2020-05-24T13:29:25.485 INFO:root:teuthology version: 1.0.0-4413d31 2020-05-24T13:29:25.488 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087418 --description rgw/multisite/{clusters.yaml frontend/beast.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} --name kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi --owner scheduled_kchai@teuthology --lock --verbose --block -- /tmp/teuthology-worker.7zl7n2q6.tmp 2020-05-24T13:29:25.526 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-24T13:29:25.591 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087418 branch: nautilus description: rgw/multisite/{clusters.yaml frontend/beast.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} email: tchaikov@gmail.com first_in_suite: false job_id: '5087418' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi nuke-on-error: true os_type: centos overrides: admin_socket: branch: nautilus ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= rgw curl low speed time: 300 rgw data log num shards: 4 rgw md log max shards: 4 rgw sync log trim interval: 0 global: osd heartbeat grace: 40 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd fast shutdown: false osd.0: osd_max_omap_entries_per_request: 10 osd.1: osd_max_omap_entries_per_request: 1000 osd.2: osd_max_omap_entries_per_request: 10000 log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck wait-for-scrub: false ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: notcmalloc sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 rgw: compression type: random frontend: beast rgw-multisite: realm: is default: true name: test-realm zonegroups: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zonegroup zones: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zone1 - endpoints: - c2.client.0 is_default: true name: test-zone2 - endpoints: - c1.client.1 name: test-zone3 - endpoints: - c2.client.1 is_pubsub: true name: test-zone4 workunit: branch: nautilus-py3 sha1: 2e1d102f0cb696365106419cf1e42700ba50a8ff owner: scheduled_kchai@teuthology priority: 90 repo: https://github.com/ceph/ceph.git roles: - - c1.mon.a - c1.mgr.x - c1.osd.0 - c1.osd.1 - c1.osd.2 - c1.client.0 - c1.client.1 - - c2.mon.a - c2.mgr.x - c2.osd.0 - c2.osd.1 - c2.osd.2 - c2.client.0 - c2.client.1 sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 sleep_before_teardown: 0 suite: rgw suite_branch: nautilus-py3 suite_path: /home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa suite_relpath: qa suite_repo: https://github.com/tchaikov/ceph.git suite_sha1: 2e1d102f0cb696365106419cf1e42700ba50a8ff tasks: - install: null - ceph: cluster: c1 - ceph: cluster: c2 - rgw: c1.client.0: port: 8000 valgrind: - --tool=memcheck - --max-threads=1024 c1.client.1: port: 8001 valgrind: - --tool=memcheck - --max-threads=1024 c2.client.0: port: 8000 valgrind: - --tool=memcheck - --max-threads=1024 c2.client.1: port: 8001 valgrind: - --tool=memcheck - --max-threads=1024 - rgw-multisite: null - rgw-multisite-tests: config: reconfigure_delay: 60 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.27836 2020-05-24T13:29:25.592 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa; will attempt to use it 2020-05-24T13:29:25.593 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa/tasks 2020-05-24T13:29:25.593 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-05-24T13:29:25.600 INFO:teuthology.task.internal:Checking packages... 2020-05-24T13:29:25.633 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '6fe217b2dddadf1baae6a015a57f83f4283a6078' 2020-05-24T13:29:25.634 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-05-24T13:29:25.634 INFO:teuthology.packaging:ref: None 2020-05-24T13:29:25.635 INFO:teuthology.packaging:tag: None 2020-05-24T13:29:25.635 INFO:teuthology.packaging:branch: nautilus 2020-05-24T13:29:25.635 INFO:teuthology.packaging:sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:29:25.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=nautilus 2020-05-24T13:29:26.110 INFO:teuthology.task.internal:Found packages for ceph version 14.2.9-580.gccd9c04 2020-05-24T13:29:26.110 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-05-24T13:29:26.121 INFO:teuthology.task.internal:no buildpackages task found 2020-05-24T13:29:26.122 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-05-24T13:29:26.143 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-05-24T13:29:26.144 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-24T13:29:26.242 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:36.267 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:46.283 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:56.311 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:06.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:16.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:26.375 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:36.394 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:46.414 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:56.442 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:31:06.469 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:31:16.489 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:31:26.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:31:36.523 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:31:46.540 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:31:56.560 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_kchai@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087418'} 2020-05-24T13:31:56.603 DEBUG:teuthology.lock.ops:locked ubuntu@smithi019.front.sepia.ceph.com, ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:31:56.671 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1923e059d8>, signals=[15]) 2020-05-24T13:31:56.883 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of centos 7.6 2020-05-24T13:31:56.888 INFO:teuthology.provision.fog.smithi193:Scheduling deploy of centos 7.6 2020-05-24T13:31:57.275 INFO:teuthology.orchestra.console:Power off smithi193 2020-05-24T13:31:57.277 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-05-24T13:31:57.298 INFO:teuthology.orchestra.console:Power off smithi019 2020-05-24T13:31:57.299 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-05-24T13:31:57.310 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:31:57.323 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:01.338 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:01.351 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:05.455 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:05.466 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:05.668 INFO:teuthology.orchestra.console:Power off for smithi193 completed 2020-05-24T13:32:05.681 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2020-05-24T13:32:05.770 INFO:teuthology.orchestra.console:Power on smithi193 2020-05-24T13:32:05.770 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-05-24T13:32:05.783 INFO:teuthology.orchestra.console:Power on smithi019 2020-05-24T13:32:05.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-05-24T13:32:05.796 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:05.812 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:09.840 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:09.854 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:13.959 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:13.974 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:14.176 INFO:teuthology.orchestra.console:Power on for smithi193 completed 2020-05-24T13:32:14.187 INFO:teuthology.orchestra.console:Power on for smithi019 completed 2020-05-24T13:32:14.278 INFO:teuthology.provision.fog.smithi193:Waiting for deploy to finish 2020-05-24T13:32:14.288 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2020-05-24T13:34:30.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:39.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:45.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:48.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:54.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:57.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:04.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:07.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:13.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:16.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:22.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:25.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:31.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:34.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:40.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:43.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:52.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:01.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:17.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:17.997 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-05-24T13:36:18.000 INFO:teuthology.orchestra.run.smithi193:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-05-24T13:36:33.503 INFO:teuthology.provision.fog.smithi193:Node is ready 2020-05-24T13:36:33.504 INFO:teuthology.orchestra.run.smithi193:> hostname 2020-05-24T13:36:33.591 INFO:teuthology.orchestra.run.smithi193.stdout:smithi125 2020-05-24T13:36:33.592 INFO:teuthology.orchestra.run.smithi193:> grep smithi125 /etc/hosts 2020-05-24T13:36:33.649 INFO:teuthology.orchestra.run.smithi193.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2020-05-24T13:36:33.650 INFO:teuthology.orchestra.run.smithi193:> sudo hostname smithi193 2020-05-24T13:36:33.803 INFO:teuthology.orchestra.run.smithi193:> sudo sed -i -e 's/smithi125/smithi193/g' /etc/hosts 2020-05-24T13:36:33.889 INFO:teuthology.orchestra.run.smithi193:> sudo sed -i -e 's/172.21.15.125/172.21.15.193/g' /etc/hosts 2020-05-24T13:36:33.965 INFO:teuthology.orchestra.run.smithi193:> sudo sed -i -e 's/smithi125/smithi193/g' /etc/hostname 2020-05-24T13:36:34.032 INFO:teuthology.orchestra.run.smithi193:> sudo hostname smithi193 2020-05-24T13:36:34.091 INFO:teuthology.provision.fog.smithi193:Deploy complete! 2020-05-24T13:36:34.092 DEBUG:teuthology.parallel:result is None 2020-05-24T13:37:07.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:37:07.689 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-05-24T13:37:07.690 INFO:teuthology.orchestra.run.smithi019:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-05-24T13:37:08.237 INFO:teuthology.provision.fog.smithi019:Node is ready 2020-05-24T13:37:08.238 INFO:teuthology.orchestra.run.smithi019:> hostname 2020-05-24T13:37:08.326 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2020-05-24T13:37:08.327 INFO:teuthology.orchestra.run.smithi019:> grep smithi019.front.sepia.ceph.com /etc/hosts 2020-05-24T13:37:08.404 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2020-05-24T13:37:08.404 INFO:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2020-05-24T13:37:08.565 INFO:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hosts 2020-05-24T13:37:08.679 INFO:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/172.21.15.19/172.21.15.19/g' /etc/hosts 2020-05-24T13:37:08.765 INFO:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hostname 2020-05-24T13:37:08.843 INFO:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2020-05-24T13:37:08.913 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2020-05-24T13:37:08.914 DEBUG:teuthology.parallel:result is None 2020-05-24T13:37:09.067 INFO:teuthology.lock.ops:Checking smithi019.front.sepia.ceph.com 2020-05-24T13:37:09.068 INFO:teuthology.lock.ops:New key found. Updating... 2020-05-24T13:37:09.089 INFO:teuthology.lock.ops:Checking smithi193.front.sepia.ceph.com 2020-05-24T13:37:09.089 INFO:teuthology.lock.ops:New key found. Updating... 2020-05-24T13:37:09.111 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: set os type and version on server 2020-05-24T13:37:09.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:37:09.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:37:09.205 INFO:teuthology.orchestra.run.smithi019:> true 2020-05-24T13:37:09.603 INFO:teuthology.orchestra.run.smithi019:> uname -m 2020-05-24T13:37:09.682 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2020-05-24T13:37:09.682 INFO:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2020-05-24T13:37:09.751 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Linux" 2020-05-24T13:37:09.751 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="7 (Core)" 2020-05-24T13:37:09.751 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2020-05-24T13:37:09.751 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:37:09.752 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="7" 2020-05-24T13:37:09.752 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:37:09.752 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2020-05-24T13:37:09.752 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:37:09.752 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:37:09.753 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:37:09.753 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:09.753 INFO:teuthology.orchestra.run.smithi019.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:37:09.753 INFO:teuthology.orchestra.run.smithi019.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:37:09.753 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:37:09.753 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:37:09.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:09.755 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2020-05-24T13:37:09.775 INFO:teuthology.lock.ops:Updating [smithi193.front.sepia.ceph.com]: set os type and version on server 2020-05-24T13:37:09.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:37:09.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:37:09.924 INFO:teuthology.orchestra.run.smithi193:> true 2020-05-24T13:37:10.339 INFO:teuthology.orchestra.run.smithi193:> uname -m 2020-05-24T13:37:10.429 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2020-05-24T13:37:10.429 INFO:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2020-05-24T13:37:10.503 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Linux" 2020-05-24T13:37:10.503 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="7 (Core)" 2020-05-24T13:37:10.503 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2020-05-24T13:37:10.504 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:37:10.504 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="7" 2020-05-24T13:37:10.504 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:37:10.504 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2020-05-24T13:37:10.504 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:37:10.505 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:37:10.505 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:37:10.505 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:37:10.505 INFO:teuthology.orchestra.run.smithi193.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:37:10.506 INFO:teuthology.orchestra.run.smithi193.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:37:10.506 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:37:10.506 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:37:10.506 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:37:10.507 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2020-05-24T13:37:10.522 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-05-24T13:37:10.553 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi019.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnzKzvSvSSWhp8CQJE68ha/aIH8OK1pi/nONXFErfC78oXKR8ZNIhgv5FDzind/CpyduLhpPcJnw+zH2e3jwHhi0e6YEGVQEIBL8pr0+Sf0fZ9+jp4nBWKF0ZTbmy2hIidWRC4S7UCmFKYiZuyTa+FKyV0YcMB0ZRMa21Y73JVd3q01kJhd9i7F0PkutoM0SzMDC9xTmXxsH60VFXIs+um4giQplsiL5QZftzbKyyDw9ayPtMUOhR9AeI7YfmETAymVal3Xy5mzAPJIhmc0lPyQoDwYz1A2+e0VKUtb56uvbKvUBLS5qdoUZhxTLGhR80urGGYmCAeY+kza7R9eVkP smithi193.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZVvdI2EmeM5Uc9ruLPP62O+Nj5/XERWhHEup3D2vviYzaLHVDcmOrC9OzuCxuSIi9SWsopYQUWD3ZekNEhKP0nqg2LVosDApxzBJ7c6gdkDMrwx6bm2uDVZVt/f6ghtUjnScO9SQeGFwpa5lkp3sxBBSkymUBAFklUc5qHzYdYpa3GhVtk7aeBOHN1de5nra+K2r94YhtOeSpXXKZybWPPenBWzRXZ8EoAkbgJPistYN1TAtCs6/54caTlA814tZ5akAMXihxO+qtUgdomjlJvasVtzTdrFHBEVhnQzt9Fd68/P9MicuvkffceblmK/n4xEjvWJttOVMAubyjtRvh 2020-05-24T13:37:10.554 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-24T13:37:10.599 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-05-24T13:37:10.609 INFO:teuthology.task.internal:Saving configuration 2020-05-24T13:37:10.622 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-05-24T13:37:10.635 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-05-24T13:37:10.651 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-05-24 13:31:56.567615', 'description': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087418', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRl4VGCSN7EIrldDD3kIVoGmelZw7gY1Bo5GYYPqryLPnUmeidGYDdGgwnG20T7tpe6zX5e8E/20G/oPbHcgrs=', 'os_version': '7', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi019.front.sepia.ceph.com'} 2020-05-24T13:37:10.664 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-05-24 13:31:56.567635', 'description': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087418', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoNxmV6D8BrqTMjaE4KzIubgNZvOfDRNgHe4bggc7sGOMboMACRfmxu5vYb9ZyCCk5QmlNa3YvdK+AOjn7ydeI=', 'os_version': '7', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi193.front.sepia.ceph.com'} 2020-05-24T13:37:10.664 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-05-24T13:37:10.674 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2020-05-24T13:37:10.675 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2020-05-24T13:37:10.675 INFO:teuthology.run_tasks:Running task console_log... 2020-05-24T13:37:10.753 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1923de4c80>, signals=[15]) 2020-05-24T13:37:10.754 INFO:teuthology.run_tasks:Running task internal.connect... 2020-05-24T13:37:10.767 INFO:teuthology.task.internal:Opening connections... 2020-05-24T13:37:10.768 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:37:10.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:37:10.876 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:37:10.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:37:10.989 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-05-24T13:37:11.000 INFO:teuthology.orchestra.run.smithi019:> uname -m 2020-05-24T13:37:11.227 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2020-05-24T13:37:11.228 INFO:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2020-05-24T13:37:11.304 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Linux" 2020-05-24T13:37:11.304 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="7 (Core)" 2020-05-24T13:37:11.305 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2020-05-24T13:37:11.305 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:37:11.305 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="7" 2020-05-24T13:37:11.305 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:37:11.306 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2020-05-24T13:37:11.306 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:37:11.306 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:37:11.306 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:37:11.306 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:11.306 INFO:teuthology.orchestra.run.smithi019.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:37:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:37:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:37:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:37:11.307 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:11.308 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2020-05-24T13:37:11.332 INFO:teuthology.orchestra.run.smithi193:> uname -m 2020-05-24T13:37:11.454 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2020-05-24T13:37:11.454 INFO:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2020-05-24T13:37:11.524 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Linux" 2020-05-24T13:37:11.524 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="7 (Core)" 2020-05-24T13:37:11.524 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2020-05-24T13:37:11.525 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:37:11.525 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="7" 2020-05-24T13:37:11.525 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:37:11.526 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2020-05-24T13:37:11.526 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:37:11.526 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:37:11.527 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:37:11.527 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:37:11.527 INFO:teuthology.orchestra.run.smithi193.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:37:11.528 INFO:teuthology.orchestra.run.smithi193.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:37:11.528 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:37:11.528 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:37:11.529 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:37:11.529 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2020-05-24T13:37:11.573 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-05-24T13:37:11.591 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-05-24T13:37:11.601 INFO:teuthology.task.internal:Checking for old test directory... 2020-05-24T13:37:11.602 INFO:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2020-05-24T13:37:11.607 INFO:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2020-05-24T13:37:11.696 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-05-24T13:37:11.705 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-05-24T13:37:11.705 INFO:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2020-05-24T13:37:11.709 INFO:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2020-05-24T13:37:11.801 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-05-24T13:37:11.846 INFO:teuthology.run_tasks:Running task kernel... 2020-05-24T13:37:11.878 DEBUG:teuthology.task.kernel:normalized config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c1.client.0': {'kdb': True, 'sha1': 'distro'}, 'c1.client.1': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c2.client.0': {'kdb': True, 'sha1': 'distro'}, 'c2.client.1': {'kdb': True, 'sha1': 'distro'}} 2020-05-24T13:37:11.879 INFO:teuthology.task.kernel:config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-05-24T13:37:11.880 INFO:teuthology.orchestra.run.smithi019:> uname -r 2020-05-24T13:37:11.968 INFO:teuthology.orchestra.run.smithi019.stdout:3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:37:11.969 INFO:teuthology.task.kernel:Running kernel on smithi019: 3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:37:11.969 INFO:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2020-05-24T13:37:12.969 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:37:13.075 INFO:teuthology.orchestra.run.smithi019.stdout:Determining fastest mirrors 2020-05-24T13:37:13.741 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:37:13.742 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:37:13.743 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:37:13.744 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:37:18.533 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:37:18.534 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:37:18.534 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package kernel.x86_64 0:3.10.0-1127.8.2.el7 will be installed 2020-05-24T13:37:19.857 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:37:20.751 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:20.751 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:37:20.752 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:20.752 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:37:20.753 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2020-05-24T13:37:20.753 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:37:20.753 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:37:20.753 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 3.10.0-1127.8.2.el7 updates 50 M 2020-05-24T13:37:20.753 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:20.754 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:37:20.754 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:37:20.754 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2020-05-24T13:37:20.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:37:20.754 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 50 M 2020-05-24T13:37:20.754 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 64 M 2020-05-24T13:37:20.755 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:37:20.755 INFO:teuthology.orchestra.run.smithi019.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:37:22.207 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:37:22.404 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:37:23.284 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:37:23.285 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:38:04.320 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:38:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:38:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:38:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:38:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: kernel.x86_64 0:3.10.0-1127.8.2.el7 2020-05-24T13:38:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:38:04.818 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:38:04.867 INFO:teuthology.orchestra.run.smithi019:> rpm -q kernel --last | head -n 1 2020-05-24T13:38:05.269 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:37:27 PM UTC 2020-05-24T13:38:05.271 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:05.272 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:05.273 INFO:teuthology.orchestra.run.smithi193:> uname -r 2020-05-24T13:38:05.381 INFO:teuthology.orchestra.run.smithi193.stdout:3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:38:05.382 INFO:teuthology.task.kernel:Running kernel on smithi193: 3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:38:05.382 INFO:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2020-05-24T13:38:06.310 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:38:06.415 INFO:teuthology.orchestra.run.smithi193.stdout:Determining fastest mirrors 2020-05-24T13:38:06.760 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:38:06.766 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:38:06.767 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:38:06.767 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:38:11.948 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:38:11.949 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:38:11.949 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package kernel.x86_64 0:3.10.0-1127.8.2.el7 will be installed 2020-05-24T13:38:13.237 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:38:14.023 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:14.024 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:38:14.042 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:14.042 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:38:14.042 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2020-05-24T13:38:14.042 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:38:14.043 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:38:14.043 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 3.10.0-1127.8.2.el7 updates 50 M 2020-05-24T13:38:14.043 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:14.043 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:38:14.043 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:38:14.044 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2020-05-24T13:38:14.044 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:14.044 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 50 M 2020-05-24T13:38:14.044 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 64 M 2020-05-24T13:38:14.044 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:38:14.050 INFO:teuthology.orchestra.run.smithi193.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:38:15.974 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:38:16.173 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:38:17.057 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:38:17.058 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:38:54.335 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:38:55.013 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:38:55.014 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:55.014 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:38:55.014 INFO:teuthology.orchestra.run.smithi193.stdout: kernel.x86_64 0:3.10.0-1127.8.2.el7 2020-05-24T13:38:55.014 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:55.015 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:38:55.062 INFO:teuthology.orchestra.run.smithi193:> rpm -q kernel --last | head -n 1 2020-05-24T13:38:55.458 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:38:19 PM UTC 2020-05-24T13:38:55.461 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:55.462 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:55.463 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-05-24T13:38:55.463 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-05-24T13:38:55.464 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-05-24T13:38:55.464 INFO:teuthology.task.kernel:Installing distro kernel on c1.mon.a... 2020-05-24T13:38:55.465 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2020-05-24T13:38:55.465 INFO:teuthology.orchestra.run.smithi019:> rpm -q kernel --last | head -n 1 2020-05-24T13:38:55.839 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:37:27 PM UTC 2020-05-24T13:38:55.841 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:55.841 INFO:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2020-05-24T13:38:56.008 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2020-05-24T13:38:56.009 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2020-05-24T13:38:56.009 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.02 2020-05-24T13:38:56.009 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 0.76.el7.centos 2020-05-24T13:38:56.009 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2020-05-24T13:38:56.009 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Tue 11 Feb 2020 03:34:40 PM UTC 2020-05-24T13:38:56.010 INFO:teuthology.orchestra.run.smithi019.stdout:Group : System Environment/Base 2020-05-24T13:38:56.010 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 10080361 2020-05-24T13:38:56.010 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2020-05-24T13:38:56.010 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2020-05-24T13:38:56.010 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2020-05-24T13:38:56.011 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2020-05-24T13:38:56.011 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : kbuilder.bsys.centos.org 2020-05-24T13:38:56.011 INFO:teuthology.orchestra.run.smithi019.stdout:Relocations : (not relocatable) 2020-05-24T13:38:56.011 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : CentOS BuildSystem 2020-05-24T13:38:56.011 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2020-05-24T13:38:56.012 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2020-05-24T13:38:56.012 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2020-05-24T13:38:56.012 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2020-05-24T13:38:56.012 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:38:56.013 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2020-05-24T13:38:56.013 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2020-05-24T13:38:56.013 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2020-05-24T13:38:56.014 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2020-05-24T13:38:56.014 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:38:56.014 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2020-05-24T13:38:56.016 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2020-05-24T13:38:56.016 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:56.017 INFO:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2020-05-24T13:38:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2020-05-24T13:38:56.673 INFO:teuthology.orchestra.run.smithi019.stderr:Found linux image: /boot/vmlinuz-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:56.689 INFO:teuthology.orchestra.run.smithi019.stderr:Found initrd image: /boot/initramfs-3.10.0-1127.8.2.el7.x86_64.img 2020-05-24T13:38:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:38:57.049 INFO:teuthology.orchestra.run.smithi019.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img 2020-05-24T13:38:57.092 INFO:teuthology.orchestra.run.smithi019.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2020-05-24T13:38:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2020-05-24T13:38:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:Found linux image: /boot/vmlinuz-0-rescue-11867a3080154962a87601c586b1198d 2020-05-24T13:38:57.156 INFO:teuthology.orchestra.run.smithi019.stderr:Found initrd image: /boot/initramfs-0-rescue-11867a3080154962a87601c586b1198d.img 2020-05-24T13:38:58.480 INFO:teuthology.orchestra.run.smithi019.stderr:done 2020-05-24T13:38:58.482 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:38:58.543 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.vIWVnaQwXB 2020-05-24T13:38:58.543 INFO:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vIWVnaQwXB 2020-05-24T13:38:58.611 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.vIWVnaQwXB 2020-05-24T13:38:58.764 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.vIWVnaQwXB is 6KB 2020-05-24T13:38:58.839 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.vIWVnaQwXB 2020-05-24T13:38:58.904 INFO:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 0 2020-05-24T13:38:59.085 INFO:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2020-05-24T13:38:59.088 INFO:teuthology.task.kernel:Installing distro kernel on c2.mon.a... 2020-05-24T13:38:59.089 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi193.front.sepia.ceph.com, path=None, version=distro) 2020-05-24T13:38:59.089 INFO:teuthology.orchestra.run.smithi193:> rpm -q kernel --last | head -n 1 2020-05-24T13:38:59.452 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:38:19 PM UTC 2020-05-24T13:38:59.454 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:59.454 INFO:teuthology.orchestra.run.smithi193:> sudo rpm -qi grub2-tools 2020-05-24T13:38:59.612 INFO:teuthology.orchestra.run.smithi193.stdout:Name : grub2-tools 2020-05-24T13:38:59.612 INFO:teuthology.orchestra.run.smithi193.stdout:Epoch : 1 2020-05-24T13:38:59.612 INFO:teuthology.orchestra.run.smithi193.stdout:Version : 2.02 2020-05-24T13:38:59.612 INFO:teuthology.orchestra.run.smithi193.stdout:Release : 0.76.el7.centos 2020-05-24T13:38:59.613 INFO:teuthology.orchestra.run.smithi193.stdout:Architecture: x86_64 2020-05-24T13:38:59.613 INFO:teuthology.orchestra.run.smithi193.stdout:Install Date: Tue 11 Feb 2020 03:34:40 PM UTC 2020-05-24T13:38:59.613 INFO:teuthology.orchestra.run.smithi193.stdout:Group : System Environment/Base 2020-05-24T13:38:59.613 INFO:teuthology.orchestra.run.smithi193.stdout:Size : 10080361 2020-05-24T13:38:59.613 INFO:teuthology.orchestra.run.smithi193.stdout:License : GPLv3+ 2020-05-24T13:38:59.614 INFO:teuthology.orchestra.run.smithi193.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2020-05-24T13:38:59.614 INFO:teuthology.orchestra.run.smithi193.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2020-05-24T13:38:59.614 INFO:teuthology.orchestra.run.smithi193.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2020-05-24T13:38:59.614 INFO:teuthology.orchestra.run.smithi193.stdout:Build Host : kbuilder.bsys.centos.org 2020-05-24T13:38:59.614 INFO:teuthology.orchestra.run.smithi193.stdout:Relocations : (not relocatable) 2020-05-24T13:38:59.614 INFO:teuthology.orchestra.run.smithi193.stdout:Packager : CentOS BuildSystem 2020-05-24T13:38:59.615 INFO:teuthology.orchestra.run.smithi193.stdout:Vendor : CentOS 2020-05-24T13:38:59.615 INFO:teuthology.orchestra.run.smithi193.stdout:URL : http://www.gnu.org/software/grub/ 2020-05-24T13:38:59.615 INFO:teuthology.orchestra.run.smithi193.stdout:Summary : Support tools for GRUB. 2020-05-24T13:38:59.615 INFO:teuthology.orchestra.run.smithi193.stdout:Description : 2020-05-24T13:38:59.615 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:59.616 INFO:teuthology.orchestra.run.smithi193.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2020-05-24T13:38:59.616 INFO:teuthology.orchestra.run.smithi193.stdout:customizable bootloader with modular architecture. It supports a rich 2020-05-24T13:38:59.616 INFO:teuthology.orchestra.run.smithi193.stdout:variety of kernel formats, file systems, computer architectures and 2020-05-24T13:38:59.616 INFO:teuthology.orchestra.run.smithi193.stdout:hardware devices. 2020-05-24T13:38:59.616 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:38:59.616 INFO:teuthology.orchestra.run.smithi193.stdout:This subpackage provides tools for support of all platforms. 2020-05-24T13:38:59.617 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2020-05-24T13:38:59.619 INFO:teuthology.task.kernel:Updating grub on smithi193 to boot 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:59.619 INFO:teuthology.orchestra.run.smithi193:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2020-05-24T13:39:00.084 INFO:teuthology.orchestra.run.smithi193.stderr:Generating grub configuration file ... 2020-05-24T13:39:00.236 INFO:teuthology.orchestra.run.smithi193.stderr:Found linux image: /boot/vmlinuz-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:39:00.252 INFO:teuthology.orchestra.run.smithi193.stderr:Found initrd image: /boot/initramfs-3.10.0-1127.8.2.el7.x86_64.img 2020-05-24T13:39:00.488 INFO:teuthology.orchestra.run.smithi193.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:39:00.502 INFO:teuthology.orchestra.run.smithi193.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img 2020-05-24T13:39:00.535 INFO:teuthology.orchestra.run.smithi193.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2020-05-24T13:39:00.552 INFO:teuthology.orchestra.run.smithi193.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2020-05-24T13:39:00.580 INFO:teuthology.orchestra.run.smithi193.stderr:Found linux image: /boot/vmlinuz-0-rescue-11867a3080154962a87601c586b1198d 2020-05-24T13:39:00.588 INFO:teuthology.orchestra.run.smithi193.stderr:Found initrd image: /boot/initramfs-0-rescue-11867a3080154962a87601c586b1198d.img 2020-05-24T13:39:01.679 INFO:teuthology.orchestra.run.smithi193.stderr:done 2020-05-24T13:39:01.681 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:39:01.733 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.dtK3ozygDa 2020-05-24T13:39:01.734 INFO:teuthology.orchestra.run.smithi193:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dtK3ozygDa 2020-05-24T13:39:01.797 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.dtK3ozygDa 2020-05-24T13:39:01.912 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.dtK3ozygDa is 6KB 2020-05-24T13:39:01.967 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.dtK3ozygDa 2020-05-24T13:39:02.013 INFO:teuthology.orchestra.run.smithi193:> sudo grub2-set-default 0 2020-05-24T13:39:02.225 INFO:teuthology.orchestra.run.smithi193:> sudo shutdown -r now 2020-05-24T13:39:32.258 INFO:teuthology.misc:Re-opening connections... 2020-05-24T13:39:32.259 INFO:teuthology.misc:trying to connect to ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:39:32.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:39:32.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:39:50.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2020-05-24T13:39:50.624 INFO:teuthology.misc:trying to connect to ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:39:50.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:39:50.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:40:50.629 DEBUG:teuthology.orchestra.remote:timed out 2020-05-24T13:40:50.631 DEBUG:teuthology.misc:waited 78.37221884727478 2020-05-24T13:40:51.632 INFO:teuthology.misc:trying to connect to ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:40:51.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:40:51.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:40:51.751 INFO:teuthology.orchestra.run.smithi019:> true 2020-05-24T13:40:52.163 DEBUG:teuthology.misc:waited 79.90359115600586 2020-05-24T13:40:53.168 INFO:teuthology.misc:trying to connect to ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:40:53.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:40:53.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:40:53.267 INFO:teuthology.orchestra.run.smithi193:> true 2020-05-24T13:40:53.675 DEBUG:teuthology.misc:waited 81.41619777679443 2020-05-24T13:40:54.676 INFO:teuthology.task.kernel:Checking client c1.mon.a for new kernel version... 2020-05-24T13:40:54.676 INFO:teuthology.task.kernel:Checking kernel version of c1.mon.a, want "3.10.0-1127.8.2.el7.x86_64"... 2020-05-24T13:40:54.677 INFO:teuthology.orchestra.run.smithi019:> uname -r 2020-05-24T13:40:54.787 INFO:teuthology.orchestra.run.smithi019.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:54.788 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1127.8.2.el7.x86_64 vs 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:54.789 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-05-24T13:40:54.789 INFO:teuthology.task.kernel:Checking client c2.mon.a for new kernel version... 2020-05-24T13:40:54.789 INFO:teuthology.task.kernel:Checking kernel version of c2.mon.a, want "3.10.0-1127.8.2.el7.x86_64"... 2020-05-24T13:40:54.790 INFO:teuthology.orchestra.run.smithi193:> uname -r 2020-05-24T13:40:55.185 INFO:teuthology.orchestra.run.smithi193.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:55.185 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1127.8.2.el7.x86_64 vs 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:55.186 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-05-24T13:40:56.186 INFO:teuthology.task.kernel:Enabling kdb on c1.mon.a... 2020-05-24T13:40:56.187 INFO:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:40:56.359 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2020-05-24T13:40:56.379 INFO:teuthology.task.kernel:Enabling kdb on c2.mon.a... 2020-05-24T13:40:56.379 INFO:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:40:56.664 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2020-05-24T13:40:56.676 INFO:teuthology.run_tasks:Running task internal.base... 2020-05-24T13:40:56.699 INFO:teuthology.task.internal:Creating test directory... 2020-05-24T13:40:56.700 INFO:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-05-24T13:40:56.704 INFO:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-05-24T13:40:56.790 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-05-24T13:40:56.804 INFO:teuthology.run_tasks:Running task internal.archive... 2020-05-24T13:40:56.817 INFO:teuthology.task.internal:Creating archive directory... 2020-05-24T13:40:56.818 INFO:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-05-24T13:40:56.822 INFO:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-05-24T13:40:56.918 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-05-24T13:40:56.928 INFO:teuthology.task.internal:Enabling coredump saving... 2020-05-24T13:40:56.929 INFO:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-05-24T13:40:56.933 INFO:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-05-24T13:40:57.043 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:57.050 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:57.073 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:57.079 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:57.081 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-05-24T13:40:57.094 INFO:teuthology.task.internal:Configuring sudo... 2020-05-24T13:40:57.094 INFO:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-05-24T13:40:57.098 INFO:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-05-24T13:40:57.207 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-05-24T13:40:57.224 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-05-24T13:40:57.225 INFO:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-05-24T13:40:57.230 INFO:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-05-24T13:40:57.312 INFO:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:57.423 INFO:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:57.563 INFO:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:57.679 INFO:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:57.791 INFO:teuthology.orchestra.run.smithi019:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-05-24T13:40:57.889 INFO:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:58.007 INFO:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:58.167 INFO:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:58.280 INFO:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:58.374 INFO:teuthology.orchestra.run.smithi193:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-05-24T13:40:58.455 INFO:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2020-05-24T13:40:58.459 INFO:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2020-05-24T13:40:58.602 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-05-24T13:40:58.630 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-05-24T13:40:58.674 INFO:teuthology.run_tasks:Running task internal.timer... 2020-05-24T13:40:58.692 INFO:teuthology.task.internal:Starting timer... 2020-05-24T13:40:58.693 INFO:teuthology.run_tasks:Running task kernel... 2020-05-24T13:40:58.704 DEBUG:teuthology.task.kernel:normalized config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c1.client.0': {'kdb': True, 'sha1': 'distro'}, 'c1.client.1': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c2.client.0': {'kdb': True, 'sha1': 'distro'}, 'c2.client.1': {'kdb': True, 'sha1': 'distro'}} 2020-05-24T13:40:58.705 INFO:teuthology.task.kernel:config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-05-24T13:40:58.705 INFO:teuthology.orchestra.run.smithi019:> uname -r 2020-05-24T13:40:58.806 INFO:teuthology.orchestra.run.smithi019.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:58.807 INFO:teuthology.task.kernel:Running kernel on smithi019: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:58.807 INFO:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2020-05-24T13:40:59.093 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:40:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:40:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:40:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:40:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:40:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:41:00.136 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-3.10.0-1127.8.2.el7.x86_64 already installed and latest version 2020-05-24T13:41:00.136 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do 2020-05-24T13:41:00.151 INFO:teuthology.orchestra.run.smithi019:> echo no | sudo yum reinstall kernel || true 2020-05-24T13:41:00.409 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:41:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:Skipping the running kernel: kernel-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:41:00.513 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:41:00.515 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:41:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:41:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:41:00.518 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:41:00.543 INFO:teuthology.orchestra.run.smithi019.stdout:Installed package kernel-3.10.0-1062.12.1.el7.x86_64 (from updates) not available. 2020-05-24T13:41:00.543 INFO:teuthology.orchestra.run.smithi019.stdout:Installed package kernel-3.10.0-957.el7.x86_64 (from anaconda) not available. 2020-05-24T13:41:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:Error: Nothing to do 2020-05-24T13:41:00.563 INFO:teuthology.task.kernel:Newest distro kernel already installed and running 2020-05-24T13:41:00.564 INFO:teuthology.orchestra.run.smithi193:> uname -r 2020-05-24T13:41:00.656 INFO:teuthology.orchestra.run.smithi193.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:41:00.657 INFO:teuthology.task.kernel:Running kernel on smithi193: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:41:00.657 INFO:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2020-05-24T13:41:01.739 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:41:01.836 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:41:01.851 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:41:01.859 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:41:01.859 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:41:01.859 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:41:02.932 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-3.10.0-1127.8.2.el7.x86_64 already installed and latest version 2020-05-24T13:41:02.932 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do 2020-05-24T13:41:02.953 INFO:teuthology.orchestra.run.smithi193:> echo no | sudo yum reinstall kernel || true 2020-05-24T13:41:03.172 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:41:03.312 INFO:teuthology.orchestra.run.smithi193.stderr:Skipping the running kernel: kernel-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:41:03.313 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:41:03.314 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:41:03.315 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:41:03.315 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:41:03.316 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:41:03.333 INFO:teuthology.orchestra.run.smithi193.stdout:Installed package kernel-3.10.0-1062.12.1.el7.x86_64 (from updates) not available. 2020-05-24T13:41:03.333 INFO:teuthology.orchestra.run.smithi193.stdout:Installed package kernel-3.10.0-957.el7.x86_64 (from anaconda) not available. 2020-05-24T13:41:03.334 INFO:teuthology.orchestra.run.smithi193.stderr:Error: Nothing to do 2020-05-24T13:41:03.352 INFO:teuthology.task.kernel:Newest distro kernel already installed and running 2020-05-24T13:41:03.353 INFO:teuthology.task.kernel:Enabling kdb on c1.mon.a... 2020-05-24T13:41:03.353 INFO:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:41:03.466 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2020-05-24T13:41:03.519 INFO:teuthology.task.kernel:Enabling kdb on c2.mon.a... 2020-05-24T13:41:03.520 INFO:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:41:03.631 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2020-05-24T13:41:03.678 INFO:teuthology.run_tasks:Running task pcp... 2020-05-24T13:41:03.738 INFO:teuthology.run_tasks:Running task selinux... 2020-05-24T13:41:03.758 INFO:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2020-05-24T13:41:03.935 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: [ OK ] 2020-05-24T13:41:03.939 INFO:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2020-05-24T13:41:04.114 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: [ OK ] 2020-05-24T13:41:04.116 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-05-24T13:41:04.116 INFO:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2020-05-24T13:41:04.212 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2020-05-24T13:41:04.213 INFO:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2020-05-24T13:41:04.322 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2020-05-24T13:41:04.322 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2020-05-24T13:41:04.323 INFO:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\)' 2020-05-24T13:41:04.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-24T13:41:04.447 INFO:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\)' 2020-05-24T13:41:04.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-24T13:41:04.562 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-05-24T13:41:04.563 INFO:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2020-05-24T13:41:04.684 INFO:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2020-05-24T13:41:04.801 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-05-24T13:41:04.815 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2020-05-24T13:41:04.827 INFO:teuthology.repo_utils:Fetching master from origin 2020-05-24T13:41:04.889 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-05-24T13:41:04.910 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-05-24T13:41:04.913 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-05-24T13:41:05.313 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-05-24T13:41:06.389 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-05-24T13:41:08.825 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-05-24T13:41:08.856 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:08 +0000 (0:00:00.073) 0:00:00.073 ************ 2020-05-24T13:41:08.862 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:08 +0000 (0:00:00.016) 0:00:00.089 ************ 2020-05-24T13:41:11.228 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi193.front.sepia.ceph.com] 2020-05-24T13:41:11.291 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] 2020-05-24T13:41:11.351 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:11 +0000 (0:00:02.488) 0:00:02.578 ************ 2020-05-24T13:41:11.393 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:11 +0000 (0:00:00.042) 0:00:02.620 ************ 2020-05-24T13:41:11.951 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-05-24T13:41:11.953 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:41:12.016 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:12 +0000 (0:00:00.614) 0:00:03.234 ************ 2020-05-24T13:41:12.018 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:41:12.073 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:12 +0000 (0:00:00.065) 0:00:03.300 ************ 2020-05-24T13:41:12.781 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-05-24T13:41:12.783 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-05-24T13:41:12.815 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:12 +0000 (0:00:00.741) 0:00:04.042 ************ 2020-05-24T13:41:12.829 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-05-24T13:41:12.863 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:12 +0000 (0:00:00.047) 0:00:04.089 ************ 2020-05-24T13:41:12.870 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-05-24T13:41:12.874 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:12.911 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:12 +0000 (0:00:00.048) 0:00:04.138 ************ 2020-05-24T13:41:12.918 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:12.973 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:12 +0000 (0:00:00.055) 0:00:04.193 ************ 2020-05-24T13:41:13.491 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi193.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:41:13.546 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:13 +0000 (0:00:00.577) 0:00:04.771 ************ 2020-05-24T13:41:13.554 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:41:13.594 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:13 +0000 (0:00:00.049) 0:00:04.821 ************ 2020-05-24T13:41:15.265 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327673.5629907-94207391336899/source", "state": "file", "uid": 0} 2020-05-24T13:41:15.304 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:15 +0000 (0:00:01.709) 0:00:06.531 ************ 2020-05-24T13:41:15.322 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327673.6204169-246840924755538/source", "state": "file", "uid": 0} 2020-05-24T13:41:15.364 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:15 +0000 (0:00:00.059) 0:00:06.590 ************ 2020-05-24T13:41:15.926 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-05-24T13:41:15.928 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-05-24T13:41:15.930 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-05-24T13:41:16.012 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-05-24T13:41:16.014 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-05-24T13:41:16.135 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-05-24T13:41:16.141 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-05-24T13:41:16.292 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:16 +0000 (0:00:00.923) 0:00:07.514 ************ 2020-05-24T13:41:16.328 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:16 +0000 (0:00:00.040) 0:00:07.555 ************ 2020-05-24T13:41:17.263 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi193.front.sepia.ceph.com] 2020-05-24T13:41:17.342 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. Sunday 24 May 2020 13:41:17 +0000 (0:00:01.014) 0:00:08.570 ************ 2020-05-24T13:41:18.008 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-05-24T13:41:18.009 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.167130", "end": "2020-05-24 13:41:17.971769", "rc": 0, "start": "2020-05-24 13:41:17.804639", "stderr": "", "stderr_lines": [], "stdout": "815dca2d192c9d570bf0eed74c506da390104ba6", "stdout_lines": ["815dca2d192c9d570bf0eed74c506da390104ba6"]} 2020-05-24T13:41:18.041 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:18 +0000 (0:00:00.698) 0:00:09.268 ************ 2020-05-24T13:41:18.293 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi019.front.sepia.ceph.com] 2020-05-24T13:41:18.394 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:18 +0000 (0:00:00.352) 0:00:09.621 ************ 2020-05-24T13:41:18.516 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.135730", "end": "2020-05-24 13:41:18.480711", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:41:18.344981", "stderr": "", "stderr_lines": [], "stdout": "966e9a6bfb22958c64d4123b47ddee87a2318d5a", "stdout_lines": ["966e9a6bfb22958c64d4123b47ddee87a2318d5a"]} 2020-05-24T13:41:18.566 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:18 +0000 (0:00:00.172) 0:00:09.793 ************ 2020-05-24T13:41:18.714 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-05-24T13:41:18.716 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:18.867 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:18 +0000 (0:00:00.300) 0:00:10.094 ************ 2020-05-24T13:41:18.876 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-05-24T13:41:18.882 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.184179", "end": "2020-05-24 13:41:18.775437", "rc": 0, "start": "2020-05-24 13:41:18.591258", "stderr": "", "stderr_lines": [], "stdout": "815dca2d192c9d570bf0eed74c506da390104ba6", "stdout_lines": ["815dca2d192c9d570bf0eed74c506da390104ba6"]} 2020-05-24T13:41:18.915 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:18 +0000 (0:00:00.048) 0:00:10.142 ************ 2020-05-24T13:41:18.924 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-05-24T13:41:18.926 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:18.959 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:18 +0000 (0:00:00.042) 0:00:10.185 ************ 2020-05-24T13:41:19.005 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-05-24T13:41:19.007 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.052 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.080) 0:00:10.266 ************ 2020-05-24T13:41:19.076 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-05-24T13:41:19.078 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.110 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.071) 0:00:10.337 ************ 2020-05-24T13:41:19.149 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-05-24T13:41:19.152 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.198 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.087) 0:00:10.424 ************ 2020-05-24T13:41:19.250 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-05-24T13:41:19.251 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008084", "end": "2020-05-24 13:41:19.228004", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:41:19.219920", "stderr": "", "stderr_lines": [], "stdout": "966e9a6bfb22958c64d4123b47ddee87a2318d5a", "stdout_lines": ["966e9a6bfb22958c64d4123b47ddee87a2318d5a"]} 2020-05-24T13:41:19.302 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.104) 0:00:10.529 ************ 2020-05-24T13:41:19.348 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.403 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.100) 0:00:10.629 ************ 2020-05-24T13:41:19.444 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-05-24T13:41:19.446 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.479 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.076) 0:00:10.706 ************ 2020-05-24T13:41:19.509 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* ok: [smithi193.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:41:19.551 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.071) 0:00:10.778 ************ 2020-05-24T13:41:19.561 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-05-24T13:41:19.563 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.621 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.069) 0:00:10.847 ************ 2020-05-24T13:41:19.683 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-05-24T13:41:19.684 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.719 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.098) 0:00:10.946 ************ 2020-05-24T13:41:19.757 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-05-24T13:41:19.759 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:19.799 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:19 +0000 (0:00:00.079) 0:00:11.026 ************ 2020-05-24T13:41:19.960 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-24T13:41:19.966 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-05-24T13:41:20.127 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-05-24T13:41:20.129 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:41:20.161 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:20 +0000 (0:00:00.362) 0:00:11.388 ************ 2020-05-24T13:41:20.275 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-24T13:41:20.277 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-05-24T13:41:20.572 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-05-24T13:41:20.611 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-05-24T13:41:20.907 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-05-24T13:41:20.929 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-05-24T13:41:21.244 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-05-24T13:41:21.310 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-05-24T13:41:21.575 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-05-24T13:41:21.659 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-05-24T13:41:21.892 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-05-24T13:41:22.009 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-05-24T13:41:22.334 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-05-24T13:41:22.535 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-05-24T13:41:22.676 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-05-24T13:41:22.884 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-05-24T13:41:23.242 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-05-24T13:41:23.292 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-05-24T13:41:23.559 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-05-24T13:41:23.622 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-05-24T13:41:23.924 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-05-24T13:41:23.971 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-05-24T13:41:24.270 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-05-24T13:41:24.323 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-05-24T13:41:24.325 INFO:teuthology.task.ansible.out: 2020-05-24T13:41:24.637 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-05-24T13:41:24.670 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-05-24T13:41:24.995 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-05-24T13:41:25.017 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-05-24T13:41:25.350 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-05-24T13:41:25.356 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-05-24T13:41:25.682 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-05-24T13:41:25.722 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-05-24T13:41:26.031 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-05-24T13:41:26.063 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-05-24T13:41:26.419 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-05-24T13:41:26.595 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "name": "cmeno", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-05-24T13:41:26.788 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-05-24T13:41:26.925 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-05-24T13:41:27.278 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-05-24T13:41:27.426 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "name": "cmeno", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-05-24T13:41:27.618 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-05-24T13:41:27.748 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-05-24T13:41:27.964 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-05-24T13:41:28.085 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-05-24T13:41:28.303 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-05-24T13:41:28.455 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-05-24T13:41:28.647 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-05-24T13:41:28.818 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-05-24T13:41:29.021 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-05-24T13:41:29.189 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-05-24T13:41:29.366 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-05-24T13:41:29.513 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-05-24T13:41:29.692 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-05-24T13:41:29.872 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-05-24T13:41:30.054 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-05-24T13:41:30.240 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-05-24T13:41:30.380 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-05-24T13:41:30.586 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-05-24T13:41:30.720 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-05-24T13:41:30.938 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-05-24T13:41:31.079 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-05-24T13:41:31.274 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-05-24T13:41:31.386 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-05-24T13:41:31.602 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-05-24T13:41:31.733 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-05-24T13:41:31.921 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-05-24T13:41:32.056 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-05-24T13:41:32.274 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-05-24T13:41:32.386 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-05-24T13:41:32.612 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-05-24T13:41:32.726 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-05-24T13:41:32.962 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-05-24T13:41:33.047 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-05-24T13:41:33.310 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-05-24T13:41:33.410 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-05-24T13:41:33.679 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-05-24T13:41:33.732 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:34.046 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-05-24T13:41:34.054 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-24T13:41:34.405 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-05-24T13:41:34.412 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-05-24T13:41:34.745 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:34.767 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-05-24T13:41:35.089 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-24T13:41:35.149 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-05-24T13:41:35.455 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-05-24T13:41:35.490 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-05-24T13:41:35.796 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-05-24T13:41:35.832 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-05-24T13:41:36.118 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-05-24T13:41:36.166 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-05-24T13:41:36.456 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-05-24T13:41:36.498 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-24T13:41:36.809 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-05-24T13:41:36.810 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-24T13:41:37.169 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-05-24T13:41:37.175 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-05-24T13:41:37.514 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-05-24T13:41:37.828 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-05-24T13:41:37.855 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-24T13:41:38.211 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-05-24T13:41:38.226 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-05-24T13:41:38.583 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-05-24T13:41:38.594 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-05-24T13:41:38.596 INFO:teuthology.task.ansible.out: 2020-05-24T13:41:38.933 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-05-24T13:41:38.935 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-05-24T13:41:39.260 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-05-24T13:41:39.299 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-05-24T13:41:39.609 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-05-24T13:41:39.665 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-05-24T13:41:39.953 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-05-24T13:41:39.995 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-05-24T13:41:40.304 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-05-24T13:41:40.306 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-05-24T13:41:40.640 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-05-24T13:41:40.656 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-05-24T13:41:40.987 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-05-24T13:41:41.019 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-05-24T13:41:41.347 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-05-24T13:41:41.381 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-05-24T13:41:41.693 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-05-24T13:41:41.715 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-05-24T13:41:42.042 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-05-24T13:41:42.069 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-05-24T13:41:42.364 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:42.427 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-05-24T13:41:42.719 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-05-24T13:41:42.777 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-05-24T13:41:43.034 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-05-24T13:41:43.123 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-05-24T13:41:43.398 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-05-24T13:41:43.482 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:43.765 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-05-24T13:41:43.841 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-05-24T13:41:44.078 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-05-24T13:41:44.187 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-05-24T13:41:44.426 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-05-24T13:41:44.546 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-05-24T13:41:44.783 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-05-24T13:41:44.909 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-05-24T13:41:45.113 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-05-24T13:41:45.234 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-05-24T13:41:45.445 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-05-24T13:41:45.582 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-05-24T13:41:45.795 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-05-24T13:41:45.949 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-05-24T13:41:46.105 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-05-24T13:41:46.294 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-05-24T13:41:46.451 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-05-24T13:41:46.639 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-05-24T13:41:46.796 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-05-24T13:41:46.982 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-05-24T13:41:47.138 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-05-24T13:41:47.340 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-05-24T13:41:47.448 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-05-24T13:41:47.697 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-05-24T13:41:47.791 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-05-24T13:41:48.032 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-05-24T13:41:48.125 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-05-24T13:41:48.389 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-05-24T13:41:48.474 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-05-24T13:41:48.749 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-05-24T13:41:48.844 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-05-24T13:41:49.099 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-05-24T13:41:49.185 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-05-24T13:41:49.447 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-05-24T13:41:49.543 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-05-24T13:41:49.824 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-05-24T13:41:49.891 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-05-24T13:41:50.141 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-05-24T13:41:50.245 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-05-24T13:41:50.503 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-05-24T13:41:50.608 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-05-24T13:41:50.848 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-05-24T13:41:50.941 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-05-24T13:41:51.213 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-05-24T13:41:51.280 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-05-24T13:41:51.574 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-05-24T13:41:51.617 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-05-24T13:41:51.901 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-05-24T13:41:51.946 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-05-24T13:41:52.257 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-05-24T13:41:52.279 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-05-24T13:41:52.617 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-05-24T13:41:52.660 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-05-24T13:41:52.993 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-05-24T13:41:53.005 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-05-24T13:41:53.308 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-05-24T13:41:53.390 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-05-24T13:41:53.651 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-05-24T13:41:53.747 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-05-24T13:41:53.987 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-05-24T13:41:54.082 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-05-24T13:41:54.351 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-05-24T13:41:54.394 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-05-24T13:41:54.708 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-05-24T13:41:54.730 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-05-24T13:41:55.065 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-05-24T13:41:55.077 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-05-24T13:41:55.406 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-24T13:41:55.417 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-05-24T13:41:55.771 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-05-24T13:41:55.772 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-05-24T13:41:56.093 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-05-24T13:41:56.115 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-05-24T13:41:56.407 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-05-24T13:41:56.451 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-05-24T13:41:56.740 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-24T13:41:56.803 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-05-24T13:41:57.082 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-05-24T13:41:57.144 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-05-24T13:41:57.440 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-05-24T13:41:57.486 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-05-24T13:41:57.773 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-05-24T13:41:57.784 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-05-24T13:41:58.107 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-05-24T13:41:58.130 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-05-24T13:41:58.466 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-05-24T13:41:58.633 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "name": "khiremat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-05-24T13:41:58.802 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-05-24T13:41:59.138 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-05-24T13:41:59.232 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "name": "kapandya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2020-05-24T13:41:59.234 INFO:teuthology.task.ansible.out: 2020-05-24T13:41:59.503 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-05-24T13:41:59.805 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "name": "prasad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2020-05-24T13:42:00.117 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "name": "khiremat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-05-24T13:42:00.363 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2020-05-24T13:42:00.704 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "name": "kapandya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2020-05-24T13:42:00.892 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2020-05-24T13:42:01.297 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "name": "prasad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2020-05-24T13:42:01.466 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2020-05-24T13:42:01.604 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:01 +0000 (0:00:41.443) 0:00:52.831 ************ 2020-05-24T13:42:01.645 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-05-24T13:42:01.691 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:01 +0000 (0:00:00.087) 0:00:52.918 ************ 2020-05-24T13:42:01.759 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c 2020-05-24T13:42:01.761 INFO:teuthology.task.ansible.out:68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwn 2020-05-24T13:42:01.762 INFO:teuthology.task.ansible.out:j7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2 2020-05-24T13:42:01.764 INFO:teuthology.task.ansible.out:714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat 2020-05-24T13:42:01.766 INFO:teuthology.task.ansible.out:.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "o 2020-05-24T13:42:01.766 INFO:teuthology.task.ansible.out:vpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 2020-05-24T13:42:01.768 INFO:teuthology.task.ansible.out:1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf 2020-05-24T13:42:01.769 INFO:teuthology.task.ansible.out:1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45 2020-05-24T13:42:01.771 INFO:teuthology.task.ansible.out:445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463 2020-05-24T13:42:01.773 INFO:teuthology.task.ansible.out:ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5 2020-05-24T13:42:01.774 INFO:teuthology.task.ansible.out:M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas 2020-05-24T13:42:01.776 INFO:teuthology.task.ansible.out:", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, 2020-05-24T13:42:01.778 INFO:teuthology.task.ansible.out:{"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}]}, "changed": false} 2020-05-24T13:42:01.813 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:01 +0000 (0:00:00.121) 0:00:53.040 ************ 2020-05-24T13:42:01.915 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-24T13:42:01.917 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2020-05-24T13:42:02.592 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2020-05-24T13:42:02.838 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-05-24T13:42:02.840 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com -> localhost] => {"after": "815dca2d192c9d570bf0eed74c506da390104ba6", "attempts": 1, "before": "815dca2d192c9d570bf0eed74c506da390104ba6", "changed": false, "remote_url_changed": false} 2020-05-24T13:42:02.875 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:02 +0000 (0:00:01.060) 0:00:54.100 ************ 2020-05-24T13:42:03.231 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-24T13:42:03.233 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2020-05-24T13:42:03.324 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:03.326 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-05-24T13:42:03.362 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:03 +0000 (0:00:00.486) 0:00:54.587 ************ 2020-05-24T13:42:03.394 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-05-24T13:42:03.426 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:03 +0000 (0:00:00.066) 0:00:54.653 ************ 2020-05-24T13:42:03.477 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-05-24T13:42:03.479 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-05-24T13:42:03.481 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAx 2020-05-24T13:42:03.483 INFO:teuthology.task.ansible.out:j+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8ea 2020-05-24T13:42:03.485 INFO:teuthology.task.ansible.out:c8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f261 2020-05-24T13:42:03.487 INFO:teuthology.task.ansible.out:02d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomai 2020-05-24T13:42:03.489 INFO:teuthology.task.ansible.out:wang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redha 2020-05-24T13:42:03.491 INFO:teuthology.task.ansible.out:t.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4Uk 2020-05-24T13:42:03.491 INFO:teuthology.task.ansible.out:ibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMiv 2020-05-24T13:42:03.493 INFO:teuthology.task.ansible.out:DNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "beng 2020-05-24T13:42:03.495 INFO:teuthology.task.ansible.out:land@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name" 2020-05-24T13:42:03.496 INFO:teuthology.task.ansible.out:: "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643 2020-05-24T13:42:03.498 INFO:teuthology.task.ansible.out:eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb 2020-05-24T13:42:03.500 INFO:teuthology.task.ansible.out:801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}]}, "changed": false} 2020-05-24T13:42:03.535 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:03 +0000 (0:00:00.108) 0:00:54.761 ************ 2020-05-24T13:42:03.661 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:03.663 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-05-24T13:42:04.042 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-05-24T13:42:04.180 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi019.front.sepia.ceph.com -> localhost] => {"after": "815dca2d192c9d570bf0eed74c506da390104ba6", "attempts": 1, "before": "815dca2d192c9d570bf0eed74c506da390104ba6", "changed": false, "remote_url_changed": false} 2020-05-24T13:42:04.215 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:04 +0000 (0:00:00.680) 0:00:55.442 ************ 2020-05-24T13:42:04.394 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg 2020-05-24T13:42:04.396 INFO:teuthology.task.ansible.out:1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-05-24T13:42:04.633 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-05-24T13:42:04.765 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-05-24T13:42:04.993 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-05-24T13:42:05.134 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-05-24T13:42:05.136 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-05-24T13:42:05.373 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-05-24T13:42:05.500 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-05-24T13:42:05.732 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqB 2020-05-24T13:42:05.734 INFO:teuthology.task.ansible.out:ZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-05-24T13:42:05.828 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-05-24T13:42:06.106 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-05-24T13:42:06.186 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-05-24T13:42:06.203 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:06.469 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-05-24T13:42:06.472 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-05-24T13:42:06.576 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-05-24T13:42:06.578 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-05-24T13:42:06.584 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:06.595 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:06.606 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:06.878 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-05-24T13:42:06.928 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-05-24T13:42:07.234 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-05-24T13:42:07.287 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2020-05-24T13:42:07.289 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-05-24T13:42:07.586 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-05-24T13:42:07.597 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:07.654 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-05-24T13:42:07.960 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-05-24T13:42:07.962 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:07.983 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:07.994 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:08.016 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-05-24T13:42:08.354 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-05-24T13:42:08.371 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-05-24T13:42:08.720 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2020-05-24T13:42:08.722 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-05-24T13:42:08.741 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-05-24T13:42:09.063 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-05-24T13:42:09.075 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-05-24T13:42:09.405 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-05-24T13:42:09.436 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-05-24T13:42:09.767 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-05-24T13:42:09.769 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-05-24T13:42:10.129 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-05-24T13:42:10.138 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-05-24T13:42:10.140 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-05-24T13:42:10.485 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-05-24T13:42:10.487 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-05-24T13:42:10.835 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-05-24T13:42:10.846 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-05-24T13:42:11.202 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-05-24T13:42:11.204 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-05-24T13:42:11.214 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-05-24T13:42:11.216 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-05-24T13:42:11.538 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-05-24T13:42:11.570 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-05-24T13:42:11.577 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-05-24T13:42:11.880 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-05-24T13:42:11.901 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-05-24T13:42:12.225 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-05-24T13:42:12.240 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-05-24T13:42:12.566 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-05-24T13:42:12.601 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-05-24T13:42:12.603 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-05-24T13:42:12.931 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-05-24T13:42:12.963 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-05-24T13:42:13.287 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-05-24T13:42:13.289 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-05-24T13:42:13.319 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-05-24T13:42:13.632 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-05-24T13:42:13.674 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-05-24T13:42:13.953 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-05-24T13:42:14.037 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-05-24T13:42:14.308 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-05-24T13:42:14.394 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-05-24T13:42:14.652 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-05-24T13:42:14.794 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-05-24T13:42:14.796 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-05-24T13:42:15.024 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-05-24T13:42:15.163 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-05-24T13:42:15.333 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-05-24T13:42:15.517 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-05-24T13:42:15.705 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:42:15.859 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-05-24T13:42:16.039 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-05-24T13:42:16.207 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-05-24T13:42:16.372 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-05-24T13:42:16.549 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-05-24T13:42:16.739 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-05-24T13:42:16.741 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-05-24T13:42:16.908 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-05-24T13:42:17.101 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-05-24T13:42:17.103 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-05-24T13:42:17.262 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:42:17.451 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-05-24T13:42:17.634 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-05-24T13:42:17.782 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-05-24T13:42:18.001 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-05-24T13:42:18.135 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-05-24T13:42:18.354 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-05-24T13:42:18.356 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-05-24T13:42:18.453 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-05-24T13:42:18.455 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:42:18.706 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-05-24T13:42:18.708 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-05-24T13:42:18.810 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-05-24T13:42:18.812 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-05-24T13:42:18.814 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-05-24T13:42:19.058 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-05-24T13:42:19.162 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-05-24T13:42:19.435 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-05-24T13:42:19.492 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-05-24T13:42:19.776 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-05-24T13:42:19.822 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-05-24T13:42:20.132 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-05-24T13:42:20.134 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:42:20.184 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-05-24T13:42:20.498 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-05-24T13:42:20.500 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-05-24T13:42:20.502 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-05-24T13:42:20.534 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-05-24T13:42:20.867 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-05-24T13:42:20.900 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/ 2020-05-24T13:42:20.902 INFO:teuthology.task.ansible.out:home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-05-24T13:42:21.206 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-05-24T13:42:21.258 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-05-24T13:42:21.554 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-05-24T13:42:21.618 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-05-24T13:42:21.897 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-05-24T13:42:21.991 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-05-24T13:42:21.993 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-05-24T13:42:22.251 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-05-24T13:42:22.314 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-05-24T13:42:22.596 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/ 2020-05-24T13:42:22.598 INFO:teuthology.task.ansible.out:home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-05-24T13:42:22.639 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-05-24T13:42:22.967 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-05-24T13:42:22.987 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-05-24T13:42:22.989 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-05-24T13:42:23.300 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-05-24T13:42:23.322 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-05-24T13:42:23.661 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-05-24T13:42:23.672 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-05-24T13:42:23.674 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-05-24T13:42:24.007 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-05-24T13:42:24.039 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-05-24T13:42:24.342 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:42:24.379 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-05-24T13:42:24.669 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-05-24T13:42:24.732 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-05-24T13:42:24.733 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-05-24T13:42:24.997 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-05-24T13:42:25.089 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-05-24T13:42:25.322 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-05-24T13:42:25.423 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-05-24T13:42:25.694 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-05-24T13:42:25.772 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-05-24T13:42:26.011 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-05-24T13:42:26.127 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:42:26.353 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-05-24T13:42:26.470 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-05-24T13:42:26.706 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-05-24T13:42:26.824 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-05-24T13:42:27.044 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-05-24T13:42:27.173 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-05-24T13:42:27.402 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-05-24T13:42:27.546 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-05-24T13:42:27.745 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-05-24T13:42:27.876 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-05-24T13:42:28.045 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-05-24T13:42:28.215 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-05-24T13:42:28.399 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-05-24T13:42:28.562 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-05-24T13:42:28.736 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-05-24T13:42:28.872 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-05-24T13:42:29.036 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-05-24T13:42:29.195 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-05-24T13:42:29.394 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-05-24T13:42:29.542 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-05-24T13:42:29.710 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-05-24T13:42:29.901 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-05-24T13:42:30.066 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-05-24T13:42:30.225 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-05-24T13:42:30.429 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-05-24T13:42:30.573 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-05-24T13:42:30.730 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-05-24T13:42:30.923 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-05-24T13:42:31.076 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-05-24T13:42:31.298 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-05-24T13:42:31.438 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-05-24T13:42:31.663 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-05-24T13:42:31.799 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-05-24T13:42:31.801 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-05-24T13:42:32.005 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-05-24T13:42:32.134 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-05-24T13:42:32.136 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-05-24T13:42:32.369 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-05-24T13:42:32.496 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-05-24T13:42:32.713 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-05-24T13:42:32.854 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-05-24T13:42:33.068 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-05-24T13:42:33.192 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-05-24T13:42:33.425 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-05-24T13:42:33.555 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-05-24T13:42:33.781 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-05-24T13:42:33.783 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-05-24T13:42:33.885 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-05-24T13:42:34.151 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-05-24T13:42:34.153 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-05-24T13:42:34.214 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-05-24T13:42:34.505 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-05-24T13:42:34.567 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-05-24T13:42:34.870 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-05-24T13:42:34.930 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-05-24T13:42:35.220 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-05-24T13:42:35.252 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-05-24T13:42:35.582 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-05-24T13:42:35.593 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-05-24T13:42:35.595 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:35.911 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-05-24T13:42:35.923 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-05-24T13:42:36.238 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-05-24T13:42:36.268 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-05-24T13:42:36.600 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-05-24T13:42:36.611 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-05-24T13:42:36.935 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-05-24T13:42:36.937 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-05-24T13:42:37.269 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-05-24T13:42:37.271 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:42:37.303 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-05-24T13:42:37.626 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-05-24T13:42:37.628 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-05-24T13:42:37.958 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-05-24T13:42:37.968 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-05-24T13:42:38.294 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-05-24T13:42:38.306 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-05-24T13:42:38.630 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-05-24T13:42:38.692 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-05-24T13:42:38.981 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkS3qZAsTQMObkUS9QBEpYlL7A3/4ypeRiVQil7UXrZkXVhYd8RAlvSBomOHtslz6OMuvm1GtP53apQQbLb2ziJXlogua/2ju17iyBP8KfyGUzx45S3XF6GaSi3C4gyviQ0hLhgg0HSaO92qw7lbjf731hXYBWXDwkXz/6wbYecykKHTM2xeIhTqCSDIx+fdrHmYW8tXp5ZIGxmCXgLhyNLaMXlNrsGHvLixrnUb3abJebdi+diQxWn4fx2Q+WwoAnY1uwawyeFJzmLEs3qjFWmuYSKcDJ94gHHedXyDhZN1mZOs0a0E319jtXG88/SQrX3+/OypS92Hr0FF6PQwhT root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-05-24T13:42:39.046 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-05-24T13:42:39.322 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-05-24T13:42:39.395 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-05-24T13:42:39.397 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:42:39.690 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-05-24T13:42:39.761 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-05-24T13:42:40.032 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-05-24T13:42:40.127 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-05-24T13:42:40.355 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-05-24T13:42:40.479 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-05-24T13:42:40.653 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-05-24T13:42:40.823 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-05-24T13:42:40.990 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-05-24T13:42:41.161 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkS3qZAsTQMObkUS9QBEpYlL7A3/4ypeRiVQil7UXrZkXVhYd8RAlvSBomOHtslz6OMuvm1GtP53apQQbLb2ziJXlogua/2ju17iyBP8KfyGUzx45S3XF6GaSi3C4gyviQ0hLhgg0HSaO92qw7lbjf731hXYBWXDwkXz/6wbYecykKHTM2xeIhTqCSDIx+fdrHmYW8tXp5ZIGxmCXgLhyNLaMXlNrsGHvLixrnUb3abJebdi+diQxWn4fx2Q+WwoAnY1uwawyeFJzmLEs3qjFWmuYSKcDJ94gHHedXyDhZN1mZOs0a0E319jtXG88/SQrX3+/OypS92Hr0FF6PQwhT root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-05-24T13:42:41.323 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-05-24T13:42:41.517 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-05-24T13:42:41.692 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2020-05-24T13:42:41.829 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-05-24T13:42:42.039 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-05-24T13:42:42.119 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:37.904) 0:01:33.346 ************ 2020-05-24T13:42:42.181 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:42.188 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.190 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.200 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:42.202 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682 2020-05-24T13:42:42.204 INFO:teuthology.task.ansible.out:b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.212 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.223 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.225 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.234 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.245 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.538 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-05-24T13:42:42.594 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:42.595 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-05-24T13:42:42.616 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.845 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:42.846 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-05-24T13:42:42.963 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:42.965 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-05-24T13:42:43.181 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-05-24T13:42:43.285 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:43.286 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-05-24T13:42:43.545 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:43.546 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-05-24T13:42:43.643 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:43.645 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2020-05-24T13:42:43.654 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.665 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.668 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.688 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.699 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.710 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.722 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.732 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.747 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.748 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.762 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.769 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.771 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.789 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.796 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.798 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.807 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.809 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.826 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.840 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.842 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.852 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.862 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.874 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:43.876 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.886 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.907 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.921 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.932 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.949 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.970 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.981 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.992 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:43.994 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.004 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.015 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.027 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.028 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.038 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.039 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.051 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.061 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.072 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.082 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.083 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.094 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.105 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.107 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.122 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.134 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.144 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.159 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.170 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.172 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.181 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.193 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.195 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.214 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.216 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-05-24T13:42:44.226 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:44.229 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.245 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:44.275 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:44.277 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.277 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1f 2020-05-24T13:42:44.279 INFO:teuthology.task.ansible.out:b0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.338 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:44 +0000 (0:00:02.217) 0:01:35.563 ************ 2020-05-24T13:42:44.358 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.360 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn' 2020-05-24T13:42:44.361 INFO:teuthology.task.ansible.out:: 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d0 2020-05-24T13:42:44.363 INFO:teuthology.task.ansible.out:11a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.373 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.384 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.395 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.399 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.411 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.412 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.424 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.425 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.427 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.427 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.429 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.440 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.442 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.452 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.454 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.464 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.475 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.477 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.495 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 2020-05-24T13:42:44.497 INFO:teuthology.task.ansible.out: 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.501 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.526 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.546 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.547 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.549 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.618 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:44 +0000 (0:00:00.281) 0:01:35.845 ************ 2020-05-24T13:42:44.650 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-05-24T13:42:44.658 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:44.685 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:44 +0000 (0:00:00.066) 0:01:35.911 ************ 2020-05-24T13:42:44.804 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:44.806 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-05-24T13:42:44.817 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.074 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi193.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-05-24T13:42:45.170 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-05-24T13:42:45.377 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:45.380 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-05-24T13:42:45.530 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:45.532 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-05-24T13:42:45.691 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:45.693 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-05-24T13:42:45.911 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:45.912 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfad 2020-05-24T13:42:45.914 INFO:teuthology.task.ansible.out:a28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@n 2020-05-24T13:42:45.915 INFO:teuthology.task.ansible.out:ew-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.924 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.945 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.947 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.958 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.982 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.994 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.005 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.017 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.020 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi193.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.032 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.033 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.047 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.048 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.063 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.102 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.110 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.115 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.128 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.139 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.150 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.162 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.173 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.176 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.185 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.196 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.219 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.223 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.230 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.232 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.242 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.259 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.261 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.271 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.286 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.299 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.304 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.315 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.327 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.338 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.350 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi193.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-05-24T13:42:46.361 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.380 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.391 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.416 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.425 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.456 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.458 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.470 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.479 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.489 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.500 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.511 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.532 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.534 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.545 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.556 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.567 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.569 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.585 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.603 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.619 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.630 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.639 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.650 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.662 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.672 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.683 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.698 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.708 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi193.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-05-24T13:42:46.719 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.731 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.742 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.743 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:46.759 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.770 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.771 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.784 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.786 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.801 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.849 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:02.164) 0:01:38.076 ************ 2020-05-24T13:42:46.884 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.930 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.080) 0:01:38.157 ************ 2020-05-24T13:42:47.040 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:47.042 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-05-24T13:42:47.327 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-05-24T13:42:47.613 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "gmeno", "name": "gmeno", "remove": false, "state": "absent"} 2020-05-24T13:42:47.658 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-05-24T13:42:47.947 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-05-24T13:42:48.000 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-05-24T13:42:48.262 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-05-24T13:42:48.346 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-05-24T13:42:48.609 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-05-24T13:42:48.695 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-05-24T13:42:48.948 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-05-24T13:42:49.022 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-05-24T13:42:49.024 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:49.261 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-05-24T13:42:49.366 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-05-24T13:42:49.607 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-05-24T13:42:49.923 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-05-24T13:42:49.962 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "gmeno", "name": "gmeno", "remove": false, "state": "absent"} 2020-05-24T13:42:50.293 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-05-24T13:42:50.311 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-05-24T13:42:50.624 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-05-24T13:42:50.657 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-05-24T13:42:50.942 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-05-24T13:42:51.035 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-05-24T13:42:51.284 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-05-24T13:42:51.389 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-05-24T13:42:51.623 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-05-24T13:42:51.737 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-05-24T13:42:51.966 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-05-24T13:42:52.080 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-05-24T13:42:52.286 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-05-24T13:42:52.433 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-05-24T13:42:52.603 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-05-24T13:42:52.769 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-05-24T13:42:52.935 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-05-24T13:42:53.113 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-05-24T13:42:53.271 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-05-24T13:42:53.452 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-05-24T13:42:53.608 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-05-24T13:42:53.807 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-05-24T13:42:53.910 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-05-24T13:42:54.147 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-05-24T13:42:54.242 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-05-24T13:42:54.462 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-05-24T13:42:54.576 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-05-24T13:42:54.791 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-05-24T13:42:54.873 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-05-24T13:42:55.152 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-05-24T13:42:55.211 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-05-24T13:42:55.498 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-05-24T13:42:55.564 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-05-24T13:42:55.829 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-05-24T13:42:55.900 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-05-24T13:42:56.158 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-05-24T13:42:56.230 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-05-24T13:42:56.485 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-05-24T13:42:56.569 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-05-24T13:42:56.818 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-05-24T13:42:56.880 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-05-24T13:42:57.142 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-05-24T13:42:57.215 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-05-24T13:42:57.484 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-05-24T13:42:57.550 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-05-24T13:42:57.824 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-05-24T13:42:57.962 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-05-24T13:42:58.152 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-05-24T13:42:58.227 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-05-24T13:42:58.492 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-05-24T13:42:58.555 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-05-24T13:42:58.831 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-05-24T13:42:58.917 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-05-24T13:42:59.169 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-05-24T13:42:59.267 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-05-24T13:42:59.330 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:59 +0000 (0:00:12.399) 0:01:50.557 ************ 2020-05-24T13:42:59.520 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-05-24T13:42:59.852 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-05-24T13:43:00.175 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-05-24T13:43:00.452 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "checksum": "fd7540224fb799c0bce2cee5784635eafe11ed5f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8ac82a894b7d17ffb3e18953ec0a949b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327779.3659682-272348030533824/source", "state": "file", "uid": 0} 2020-05-24T13:43:00.494 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:00 +0000 (0:00:01.163) 0:01:51.720 ************ 2020-05-24T13:43:00.527 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi019.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-05-24T13:43:00.528 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-05-24T13:43:00.529 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => { "msg": "Host smithi193.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2020-05-24T13:43:00.582 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:00 +0000 (0:00:00.089) 0:01:51.809 ************ 2020-05-24T13:43:00.623 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-05-24T13:43:00.625 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-05-24T13:43:00.657 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:00 +0000 (0:00:00.074) 0:01:51.884 ************ 2020-05-24T13:43:00.689 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-05-24T13:43:00.692 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:00.723 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:00 +0000 (0:00:00.066) 0:01:51.950 ************ 2020-05-24T13:43:00.870 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:43:00.872 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-05-24T13:43:01.143 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-05-24T13:43:01.145 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.085921", "end": "2020-05-24 13:43:01.112352", "rc": 0, "start": "2020-05-24 13:43:01.026431", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-05-24T13:43:01.177 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.453) 0:01:52.404 ************ 2020-05-24T13:43:01.207 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-05-24T13:43:01.209 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.257 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.080) 0:01:52.484 ************ 2020-05-24T13:43:01.267 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:43:01.269 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-05-24T13:43:01.290 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.325 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.067) 0:01:52.552 ************ 2020-05-24T13:43:01.366 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-05-24T13:43:01.370 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.420 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.095) 0:01:52.647 ************ 2020-05-24T13:43:01.459 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-05-24T13:43:01.461 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.495 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.074) 0:01:52.722 ************ 2020-05-24T13:43:01.549 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi019.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.590 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.095) 0:01:52.817 ************ 2020-05-24T13:43:01.623 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-05-24T13:43:01.625 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.663 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.072) 0:01:52.890 ************ 2020-05-24T13:43:01.703 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-05-24T13:43:01.710 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi193.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:01.737 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.074) 0:01:52.964 ************ 2020-05-24T13:43:01.784 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.816 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.078) 0:01:53.043 ************ 2020-05-24T13:43:01.855 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-05-24T13:43:01.856 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:01.904 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.087) 0:01:53.130 ************ 2020-05-24T13:43:01.914 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:43:01.916 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-05-24T13:43:01.973 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:00.062) 0:01:53.193 ************ 2020-05-24T13:43:01.979 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-05-24T13:43:01.981 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:02.016 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.049) 0:01:53.243 ************ 2020-05-24T13:43:02.056 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-05-24T13:43:02.057 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.100 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.084) 0:01:53.327 ************ 2020-05-24T13:43:02.152 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-05-24T13:43:02.154 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:02.200 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.100) 0:01:53.427 ************ 2020-05-24T13:43:02.253 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.286 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.086) 0:01:53.513 ************ 2020-05-24T13:43:02.326 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-05-24T13:43:02.327 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.367 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.080) 0:01:53.594 ************ 2020-05-24T13:43:02.421 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.457 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.089) 0:01:53.683 ************ 2020-05-24T13:43:02.511 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.544 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.087) 0:01:53.771 ************ 2020-05-24T13:43:02.590 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-05-24T13:43:02.594 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:02.636 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.091) 0:01:53.863 ************ 2020-05-24T13:43:02.679 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:43:02.681 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.714 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.077) 0:01:53.941 ************ 2020-05-24T13:43:02.753 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-05-24T13:43:02.755 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.787 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.073) 0:01:54.014 ************ 2020-05-24T13:43:02.827 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-05-24T13:43:02.829 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.861 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.073) 0:01:54.088 ************ 2020-05-24T13:43:02.904 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-05-24T13:43:02.906 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:02.945 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:02 +0000 (0:00:00.083) 0:01:54.171 ************ 2020-05-24T13:43:02.984 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:03.016 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.072) 0:01:54.243 ************ 2020-05-24T13:43:03.055 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-05-24T13:43:03.056 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:03.098 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.081) 0:01:54.325 ************ 2020-05-24T13:43:03.141 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-05-24T13:43:03.143 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:03.181 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.082) 0:01:54.407 ************ 2020-05-24T13:43:03.218 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-05-24T13:43:03.220 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "checksum": "fd7540224fb799c0bce2cee5784635eafe11ed5f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8ac82a894b7d17ffb3e18953ec0a949b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327782.007895-231815324508845/source", "state": "file", "uid": 0} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:03.266 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.085) 0:01:54.493 ************ 2020-05-24T13:43:03.303 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.036) 0:01:54.530 ************ 2020-05-24T13:43:03.313 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-05-24T13:43:03.315 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => { "msg": "Host smithi019.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2020-05-24T13:43:03.351 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.048) 0:01:54.578 ************ 2020-05-24T13:43:03.359 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-05-24T13:43:03.360 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:03.396 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.044) 0:01:54.622 ************ 2020-05-24T13:43:03.408 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-05-24T13:43:03.459 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.063) 0:01:54.686 ************ 2020-05-24T13:43:03.497 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-05-24T13:43:03.499 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:03.529 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.070) 0:01:54.756 ************ 2020-05-24T13:43:03.732 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-05-24T13:43:03.734 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:43:03.766 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.236) 0:01:54.993 ************ 2020-05-24T13:43:03.945 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-05-24T13:43:03.947 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.076601", "end": "2020-05-24 13:43:03.920010", "rc": 0, "start": "2020-05-24 13:43:03.843409", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-05-24T13:43:03.981 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:03 +0000 (0:00:00.215) 0:01:55.208 ************ 2020-05-24T13:43:04.023 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-05-24T13:43:04.025 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.058 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.076) 0:01:55.285 ************ 2020-05-24T13:43:04.105 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-05-24T13:43:04.108 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.142 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.084) 0:01:55.369 ************ 2020-05-24T13:43:04.183 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.217 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.075) 0:01:55.444 ************ 2020-05-24T13:43:04.261 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-05-24T13:43:04.263 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.311 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.093) 0:01:55.538 ************ 2020-05-24T13:43:04.341 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-05-24T13:43:04.346 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.386 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.075) 0:01:55.613 ************ 2020-05-24T13:43:04.425 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-05-24T13:43:04.427 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.463 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.076) 0:01:55.690 ************ 2020-05-24T13:43:04.527 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-05-24T13:43:04.529 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:04.562 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.099) 0:01:55.789 ************ 2020-05-24T13:43:04.600 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.662 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.100) 0:01:55.889 ************ 2020-05-24T13:43:04.701 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-05-24T13:43:04.707 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.750 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.084) 0:01:55.973 ************ 2020-05-24T13:43:04.776 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-05-24T13:43:04.777 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:04.810 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.063) 0:01:56.037 ************ 2020-05-24T13:43:04.852 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-05-24T13:43:04.854 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:04.893 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.082) 0:01:56.120 ************ 2020-05-24T13:43:04.913 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-05-24T13:43:04.915 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "md5sum": "5170d95301fca1f1242a0fb0a528bbcf", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327783.8258777-8542174140771/source", "state": "file", "uid": 0} 2020-05-24T13:43:04.936 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:04.971 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.077) 0:01:56.198 ************ 2020-05-24T13:43:05.001 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-05-24T13:43:05.003 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.037 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.066) 0:01:56.264 ************ 2020-05-24T13:43:05.071 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.111 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.073) 0:01:56.338 ************ 2020-05-24T13:43:05.149 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-05-24T13:43:05.151 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.188 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.076) 0:01:56.414 ************ 2020-05-24T13:43:05.230 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.268 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.080) 0:01:56.495 ************ 2020-05-24T13:43:05.303 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:05.332 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.064) 0:01:56.559 ************ 2020-05-24T13:43:05.372 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:43:05.376 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.416 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.075) 0:01:56.635 ************ 2020-05-24T13:43:05.450 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.488 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.075) 0:01:56.710 ************ 2020-05-24T13:43:05.514 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-05-24T13:43:05.516 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.549 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.065) 0:01:56.776 ************ 2020-05-24T13:43:05.589 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.622 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.073) 0:01:56.849 ************ 2020-05-24T13:43:05.656 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.696 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.074) 0:01:56.923 ************ 2020-05-24T13:43:05.747 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.780 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.084) 0:01:57.007 ************ 2020-05-24T13:43:05.819 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-05-24T13:43:05.821 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:43:05.853 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.072) 0:01:57.080 ************ 2020-05-24T13:43:05.886 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-05-24T13:43:05.888 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:05.921 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:05 +0000 (0:00:00.067) 0:01:57.148 ************ 2020-05-24T13:43:05.963 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:06.007 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:06 +0000 (0:00:00.086) 0:01:57.234 ************ 2020-05-24T13:43:06.016 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-05-24T13:43:06.017 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "md5sum": "00fe4de6cbff2433b9d92afb443fe308", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327784.9343827-125500783684655/source", "state": "file", "uid": 0} 2020-05-24T13:43:06.074 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:06 +0000 (0:00:00.066) 0:01:57.301 ************ 2020-05-24T13:43:06.358 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-05-24T13:43:06.360 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:43:06.405 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:06 +0000 (0:00:00.331) 0:01:57.632 ************ 2020-05-24T13:43:07.271 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-05-24T13:43:07.273 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "state": "file", "uid": 0} 2020-05-24T13:43:07.646 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-05-24T13:43:07.648 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "md5sum": "5170d95301fca1f1242a0fb0a528bbcf", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327786.460024-250398434828130/source", "state": "file", "uid": 0} 2020-05-24T13:43:08.282 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi193.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "state": "file", "uid": 0} 2020-05-24T13:43:08.310 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:08 +0000 (0:00:01.904) 0:01:59.537 ************ 2020-05-24T13:43:08.857 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-05-24T13:43:08.858 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "md5sum": "00fe4de6cbff2433b9d92afb443fe308", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327787.6586227-145332783908176/source", "state": "file", "uid": 0} 2020-05-24T13:43:08.918 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:08 +0000 (0:00:00.605) 0:02:00.143 ************ 2020-05-24T13:43:08.934 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:43:08.935 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum module rather than running 'yum'. If you need to use command because yum is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": "yum clean all", "delta": "0:00:00.273811", "end": "2020-05-24 13:43:08.867127", "rc": 0, "start": "2020-05-24 13:43:08.593316", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nCleaning up list of fastest mirrors", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Cleaning up list of fastest mirrors"]} 2020-05-24T13:43:08.975 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:08 +0000 (0:00:00.058) 0:02:00.202 ************ 2020-05-24T13:43:09.026 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-05-24T13:43:09.028 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.068 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.092) 0:02:00.295 ************ 2020-05-24T13:43:09.097 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-05-24T13:43:09.099 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.145 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.077) 0:02:00.372 ************ 2020-05-24T13:43:09.179 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-05-24T13:43:09.181 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.214 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.067) 0:02:00.440 ************ 2020-05-24T13:43:09.253 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-05-24T13:43:09.255 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.286 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.072) 0:02:00.513 ************ 2020-05-24T13:43:09.343 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-05-24T13:43:09.345 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.386 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.098) 0:02:00.611 ************ 2020-05-24T13:43:09.414 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-05-24T13:43:09.416 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.447 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.063) 0:02:00.674 ************ 2020-05-24T13:43:09.475 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.509 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.061) 0:02:00.736 ************ 2020-05-24T13:43:09.536 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-05-24T13:43:09.538 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:09.573 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:09 +0000 (0:00:00.063) 0:02:00.800 ************ 2020-05-24T13:43:09.891 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-05-24T13:43:09.893 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "state": "file", "uid": 0} 2020-05-24T13:43:10.817 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "state": "file", "uid": 0} 2020-05-24T13:43:10.865 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:10 +0000 (0:00:01.290) 0:02:02.090 ************ 2020-05-24T13:43:11.461 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:43:11.463 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": "yum clean all", "delta": "0:00:00.271158", "end": "2020-05-24 13:43:11.432011", "rc": 0, "start": "2020-05-24 13:43:11.160853", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nCleaning up list of fastest mirrors", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Cleaning up list of fastest mirrors"]} 2020-05-24T13:43:11.503 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:11 +0000 (0:00:00.640) 0:02:02.730 ************ 2020-05-24T13:43:11.534 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-05-24T13:43:11.536 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.637313", "end": "2020-05-24 13:43:11.500483", "rc": 0, "start": "2020-05-24 13:43:09.863170", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:43:11.574 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:11 +0000 (0:00:00.071) 0:02:02.801 ************ 2020-05-24T13:43:11.585 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:11.621 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:11 +0000 (0:00:00.046) 0:02:02.848 ************ 2020-05-24T13:43:11.662 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-05-24T13:43:11.664 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:11.714 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:11 +0000 (0:00:00.092) 0:02:02.941 ************ 2020-05-24T13:43:11.754 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-05-24T13:43:11.756 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:11.794 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:11 +0000 (0:00:00.080) 0:02:03.021 ************ 2020-05-24T13:43:11.840 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-05-24T13:43:11.842 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:11.881 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:11 +0000 (0:00:00.086) 0:02:03.108 ************ 2020-05-24T13:43:11.933 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:11.983 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:11 +0000 (0:00:00.101) 0:02:03.210 ************ 2020-05-24T13:43:12.044 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-05-24T13:43:12.046 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:12.081 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.096) 0:02:03.306 ************ 2020-05-24T13:43:12.123 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-05-24T13:43:12.125 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:12.158 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.076) 0:02:03.383 ************ 2020-05-24T13:43:12.195 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-05-24T13:43:12.196 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:12.256 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.100) 0:02:03.483 ************ 2020-05-24T13:43:12.421 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2020-05-24T13:43:12.458 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.201) 0:02:03.685 ************ 2020-05-24T13:43:12.498 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-05-24T13:43:12.506 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:12.539 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.081) 0:02:03.766 ************ 2020-05-24T13:43:12.580 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-05-24T13:43:12.582 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:12.628 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.088) 0:02:03.855 ************ 2020-05-24T13:43:12.673 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:12.706 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.077) 0:02:03.933 ************ 2020-05-24T13:43:12.741 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:12.784 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:12 +0000 (0:00:00.077) 0:02:04.010 ************ 2020-05-24T13:43:13.535 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.936786", "end": "2020-05-24 13:43:13.500494", "rc": 0, "start": "2020-05-24 13:43:12.563708", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:43:13.571 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:13 +0000 (0:00:00.787) 0:02:04.798 ************ 2020-05-24T13:43:13.755 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-05-24T13:43:13.757 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-05-24T13:43:13.792 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:13 +0000 (0:00:00.220) 0:02:05.019 ************ 2020-05-24T13:43:14.117 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-05-24T13:43:14.119 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2020-05-24T13:43:14.175 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:14 +0000 (0:00:00.382) 0:02:05.401 ************ 2020-05-24T13:43:14.205 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-05-24T13:43:14.207 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:14.241 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:14 +0000 (0:00:00.066) 0:02:05.468 ************ 2020-05-24T13:43:14.269 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-05-24T13:43:14.271 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:14.302 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:14 +0000 (0:00:00.060) 0:02:05.529 ************ 2020-05-24T13:43:14.342 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-05-24T13:43:14.344 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:14.376 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:14 +0000 (0:00:00.074) 0:02:05.603 ************ 2020-05-24T13:43:14.425 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-05-24T13:43:14.427 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:14.463 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:14 +0000 (0:00:00.086) 0:02:05.690 ************ 2020-05-24T13:43:15.443 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-05-24T13:43:15.496 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:15 +0000 (0:00:01.033) 0:02:06.723 ************ 2020-05-24T13:43:27.458 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-05-24T13:43:27.461 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["smartmontools", "1:7.0-2.el7.x86_64 from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package smartmontools.x86_64 1:7.0-1.el7_7.1 will be updated\n---> Package smartmontools.x86_64 1:7.0-2.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n smartmontools x86_64 1:7.0-2.el7 base 546 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package\n\nTotal download size: 546 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Cleanup : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n Verifying : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Verifying : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n\nUpdated:\n smartmontools.x86_64 1:7.0-2.el7 \n\nComplete!\n"]} 2020-05-24T13:43:27.500 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:27 +0000 (0:00:12.003) 0:02:18.727 ************ 2020-05-24T13:43:28.541 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi193.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-05-24T13:43:28.909 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["smartmontools", "1:7.0-2.el7.x86_64 from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package smartmontools.x86_64 1:7.0-1.el7_7.1 will be updated\n---> Package smartmontools.x86_64 1:7.0-2.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n smartmontools x86_64 1:7.0-2.el7 base 546 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package\n\nTotal download size: 546 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Cleanup : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n Verifying : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Verifying : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n\nUpdated:\n smartmontools.x86_64 1:7.0-2.el7 \n\nComplete!\n"]} 2020-05-24T13:43:28.947 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:28 +0000 (0:00:01.446) 0:02:20.173 ************ 2020-05-24T13:43:29.467 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-24T13:43:29.469 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-05-24T13:43:30.040 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-05-24T13:43:30.340 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-05-24T13:43:30.396 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:30 +0000 (0:00:01.449) 0:02:21.623 ************ 2020-05-24T13:43:30.738 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:43:30.772 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:30 +0000 (0:00:00.376) 0:02:21.999 ************ 2020-05-24T13:43:31.006 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-24T13:43:31.008 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-05-24T13:43:31.634 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-05-24T13:43:31.637 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-05-24T13:43:31.911 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-24T13:43:31.913 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-05-24T13:43:31.952 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:31 +0000 (0:00:01.179) 0:02:23.179 ************ 2020-05-24T13:43:32.279 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-05-24T13:43:32.281 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:43:32.313 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:32 +0000 (0:00:00.360) 0:02:23.540 ************ 2020-05-24T13:43:32.522 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi193.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-05-24T13:43:33.239 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-05-24T13:43:33.393 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-05-24T13:43:33.433 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:33 +0000 (0:00:01.117) 0:02:24.658 ************ 2020-05-24T13:43:33.478 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-05-24T13:43:33.480 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:33.515 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:33 +0000 (0:00:00.083) 0:02:24.742 ************ 2020-05-24T13:43:33.570 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-05-24T13:43:33.578 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:43:33.612 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:33 +0000 (0:00:00.097) 0:02:24.839 ************ 2020-05-24T13:43:33.943 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-05-24T13:43:33.945 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-05-24T13:43:33.975 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:33 +0000 (0:00:00.362) 0:02:25.202 ************ 2020-05-24T13:43:34.024 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi193.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:34.045 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:34.067 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:34.078 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:34.112 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:34 +0000 (0:00:00.135) 0:02:25.338 ************ 2020-05-24T13:43:34.194 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-05-24T13:43:34.196 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-05-24T13:43:35.128 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-05-24T13:43:35.161 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:35 +0000 (0:00:01.050) 0:02:26.388 ************ 2020-05-24T13:43:35.201 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-05-24T13:43:35.203 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:35.238 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:35 +0000 (0:00:00.071) 0:02:26.460 ************ 2020-05-24T13:43:35.283 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:43:35.316 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:35 +0000 (0:00:00.082) 0:02:26.543 ************ 2020-05-24T13:43:35.668 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-05-24T13:43:35.704 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:35 +0000 (0:00:00.388) 0:02:26.931 ************ 2020-05-24T13:43:35.757 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:35.767 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:35.778 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:35.789 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:35.833 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:35 +0000 (0:00:00.121) 0:02:27.053 ************ 2020-05-24T13:43:41.782 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-05-24T13:43:41.784 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["nagios-plugins", "2.3.3-2.el7.x86_64 from epel"], ["nrpe", "4.0.3-1.el7.x86_64 from epel"], ["nagios-common", "4.4.5-7.el7.x86_64 from epel"], ["nagios-plugins-load", "2.3.3-2.el7.x86_64 from epel"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package nagios-common.x86_64 0:4.4.3-1.el7 will be updated\n---> Package nagios-common.x86_64 0:4.4.5-7.el7 will be an update\n---> Package nagios-plugins.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nagios-plugins-load.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins-load.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nrpe.x86_64 0:3.2.1-8.el7 will be updated\n---> Package nrpe.x86_64 0:4.0.3-1.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n nagios-common x86_64 4.4.5-7.el7 epel 27 k\n nagios-plugins x86_64 2.3.3-2.el7 epel 113 k\n nagios-plugins-load x86_64 2.3.3-2.el7 epel 41 k\n nrpe x86_64 4.0.3-1.el7 epel 264 k\n\nTransaction Summary\n============================================================================== 2020-05-24T13:43:41.786 INFO:teuthology.task.ansible.out:==\nUpgrade 4 Packages\n\nTotal download size: 446 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 3.2 MB/s | 446 kB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : nagios-common-4.4.5-7.el7.x86_64 1/8 \n Updating : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Updating : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Updating : nrpe-4.0.3-1.el7.x86_64 4/8 \nwarning: /etc/nagios/nrpe.cfg created as /etc/nagios/nrpe.cfg.rpmnew\n Cleanup : nagios-plugins-load-2.3.1-1.el7.x86_64 5/8 \n Cleanup : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Cleanup : nrpe-3.2.1-8.el7.x86_64 7/8 \n Cleanup : nagios-common-4.4.3-1.el7.x86_64 8/8 \n Verifying : nrpe-4.0.3-1.el7.x86_64 1/8 \n Verifying : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Verifying : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Verifying : nagios-common-4.4.5-7.el7.x86_64 4/8 \n Verifying : nrpe-3.2.1-8.el7.x86_64 5/8 \n Verifying : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Verifying : nagios-plugins-load-2.3.1-1.el7.x86_64 7/8 \n Verifying : nagios-common-4.4.3-1.el7.x86_64 8/8 \n\nUpdated:\n nagios-common.x86_64 0:4.4.5-7.el7 nagios-plugins.x86_64 0:2.3.3-2.el7 \n nagios-plugins-load.x86_64 0:2.3.3-2.el7 nrpe.x86_64 0:4.0.3-1.el7 \n\nComplete!\n"]} 2020-05-24T13:43:41.833 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:41 +0000 (0:00:06.006) 0:02:33.060 ************ 2020-05-24T13:43:41.896 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-05-24T13:43:41.897 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:41.945 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:41 +0000 (0:00:00.112) 0:02:33.172 ************ 2020-05-24T13:43:42.937 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-05-24T13:43:42.939 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-05-24T13:43:42.982 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:42 +0000 (0:00:01.035) 0:02:34.208 ************ 2020-05-24T13:43:43.663 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-05-24T13:43:43.666 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["nagios-plugins", "2.3.3-2.el7.x86_64 from epel"], ["nrpe", "4.0.3-1.el7.x86_64 from epel"], ["nagios-common", "4.4.5-7.el7.x86_64 from epel"], ["nagios-plugins-load", "2.3.3-2.el7.x86_64 from epel"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package nagios-common.x86_64 0:4.4.3-1.el7 will be updated\n---> Package nagios-common.x86_64 0:4.4.5-7.el7 will be an update\n---> Package nagios-plugins.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nagios-plugins-load.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins-load.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nrpe.x86_64 0:3.2.1-8.el7 will be updated\n---> Package nrpe.x86_64 0:4.0.3-1.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n nagios-common x86_64 4.4.5-7.el7 epel 27 k\n nagios-plugins x86_64 2.3.3-2.el7 epel 113 k\n nagios-plugins-load x86_64 2.3.3-2.el7 epel 41 k\n nrpe x86_64 4.0.3-1.el7 epel 264 k\n\nTransaction Summary\n============================================================================== 2020-05-24T13:43:43.668 INFO:teuthology.task.ansible.out:==\nUpgrade 4 Packages\n\nTotal download size: 446 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 3.2 MB/s | 446 kB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : nagios-common-4.4.5-7.el7.x86_64 1/8 \n Updating : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Updating : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Updating : nrpe-4.0.3-1.el7.x86_64 4/8 \nwarning: /etc/nagios/nrpe.cfg created as /etc/nagios/nrpe.cfg.rpmnew\n Cleanup : nagios-plugins-load-2.3.1-1.el7.x86_64 5/8 \n Cleanup : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Cleanup : nrpe-3.2.1-8.el7.x86_64 7/8 \n Cleanup : nagios-common-4.4.3-1.el7.x86_64 8/8 \n Verifying : nrpe-4.0.3-1.el7.x86_64 1/8 \n Verifying : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Verifying : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Verifying : nagios-common-4.4.5-7.el7.x86_64 4/8 \n Verifying : nrpe-3.2.1-8.el7.x86_64 5/8 \n Verifying : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Verifying : nagios-plugins-load-2.3.1-1.el7.x86_64 7/8 \n Verifying : nagios-common-4.4.3-1.el7.x86_64 8/8 \n\nUpdated:\n nagios-common.x86_64 0:4.4.5-7.el7 nagios-plugins.x86_64 0:2.3.3-2.el7 \n nagios-plugins-load.x86_64 0:2.3.3-2.el7 nrpe.x86_64 0:4.0.3-1.el7 \n\nComplete!\n"]} 2020-05-24T13:43:43.712 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:43 +0000 (0:00:00.731) 0:02:34.939 ************ 2020-05-24T13:43:43.754 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-05-24T13:43:43.756 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:43.787 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:43 +0000 (0:00:00.075) 0:02:35.014 ************ 2020-05-24T13:43:43.914 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-05-24T13:43:43.916 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-05-24T13:43:43.966 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:43 +0000 (0:00:00.175) 0:02:35.190 ************ 2020-05-24T13:43:44.014 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi193.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:44.041 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:44 +0000 (0:00:00.078) 0:02:35.268 ************ 2020-05-24T13:43:44.359 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-05-24T13:43:44.361 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:43:44.403 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:44 +0000 (0:00:00.362) 0:02:35.630 ************ 2020-05-24T13:43:44.802 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-05-24T13:43:44.804 INFO:teuthology.task.ansible.out:fatal: [smithi193.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-05-24T13:43:44.848 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:44 +0000 (0:00:00.445) 0:02:36.075 ************ 2020-05-24T13:43:44.858 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-05-24T13:43:44.903 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:44 +0000 (0:00:00.054) 0:02:36.130 ************ 2020-05-24T13:43:44.913 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-05-24T13:43:44.914 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:44.952 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:44 +0000 (0:00:00.049) 0:02:36.179 ************ 2020-05-24T13:43:45.890 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-05-24T13:43:45.934 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:45 +0000 (0:00:00.981) 0:02:37.161 ************ 2020-05-24T13:43:45.944 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-05-24T13:43:45.946 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2020-05-24T13:43:45.986 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:45 +0000 (0:00:00.051) 0:02:37.213 ************ 2020-05-24T13:43:45.999 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-05-24T13:43:46.000 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:46.058 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:46 +0000 (0:00:00.071) 0:02:37.284 ************ 2020-05-24T13:43:46.398 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-05-24T13:43:46.400 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:43:46.437 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:46 +0000 (0:00:00.379) 0:02:37.664 ************ 2020-05-24T13:43:46.806 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-05-24T13:43:46.808 INFO:teuthology.task.ansible.out:fatal: [smithi019.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-05-24T13:43:46.853 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:46 +0000 (0:00:00.416) 0:02:38.080 ************ 2020-05-24T13:43:46.893 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-05-24T13:43:46.894 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:46.929 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:46 +0000 (0:00:00.076) 0:02:38.156 ************ 2020-05-24T13:43:46.990 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-05-24T13:43:46.991 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2020-05-24 13:43:40 UTC", "ActiveEnterTimestampMonotonic": "200477934", "ActiveExitTimestamp": "Sun 2020-05-24 13:43:40 UTC", "ActiveExitTimestampMonotonic": "200465958", "ActiveState": "active", "After": "network.target nss-lookup.target local-fs.target time-sync.target system.slice basic.target systemd-journald.socket var-run.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2020-05-24 13:43:40 UTC", "AssertTimestampMonotonic": "200476912", "Before": "multi-user.target getty@tty1.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2020-05-24 13:43:40 UTC", "ConditionTimestampMonotonic": "200476912", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "31850", "ExecMainStartTimestamp": "Sun 2020-05-24 13:43:40 UTC", "ExecMainStartTimestampMonotonic": "200477840", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/ 2020-05-24T13:43:46.993 INFO:teuthology.task.ansible.out:usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:40 UTC] ; stop_time=[n/a] ; pid=31850 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:40 UTC] ; stop_time=[Sun 2020-05-24 13:43:40 UTC] ; pid=31847 ; code=exited ; status=0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2020-05-24 13:43:40 UTC", "InactiveEnterTimestampMonotonic": "200475365", "InactiveExitTimestamp": "Sun 2020-05-24 13:43:40 UTC", "InactiveExitTimestampMonotonic": "200477934", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "31850", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", 2020-05-24T13:43:46.994 INFO:teuthology.task.ansible.out: "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2020-05-24 13:43:40 UTC", "WatchdogTimestampMonotonic": "200477902", "WatchdogUSec": "0"}} 2020-05-24T13:43:47.028 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:47 +0000 (0:00:00.098) 0:02:38.255 ************ 2020-05-24T13:43:47.342 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-05-24T13:43:47.344 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004667", "end": "2020-05-24 13:43:47.317866", "rc": 0, "start": "2020-05-24 13:43:47.313199", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-05-24T13:43:47.378 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:47 +0000 (0:00:00.349) 0:02:38.604 ************ 2020-05-24T13:43:47.860 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-05-24T13:43:47.862 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2020-05-24T13:43:47.894 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:47 +0000 (0:00:00.516) 0:02:39.121 ************ 2020-05-24T13:43:47.903 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-05-24T13:43:47.905 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2020-05-24T13:43:47.940 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:47 +0000 (0:00:00.045) 0:02:39.167 ************ 2020-05-24T13:43:48.317 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-05-24T13:43:48.320 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2020-05-24 13:43:42 UTC", "ActiveEnterTimestampMonotonic": "214829544", "ActiveExitTimestamp": "Sun 2020-05-24 13:43:42 UTC", "ActiveExitTimestampMonotonic": "214818410", "ActiveState": "active", "After": "system.slice nss-lookup.target var-run.mount systemd-journald.socket local-fs.target basic.target network.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2020-05-24 13:43:42 UTC", "AssertTimestampMonotonic": "214828927", "Before": "getty@tty1.service xdm.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2020-05-24 13:43:42 UTC", "ConditionTimestampMonotonic": "214828927", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "31484", "ExecMainStartTimestamp": "Sun 2020-05-24 13:43:42 UTC", "ExecMainStartTimestampMonotonic": "214829489", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/ 2020-05-24T13:43:48.321 INFO:teuthology.task.ansible.out:usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:42 UTC] ; stop_time=[n/a] ; pid=31484 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:42 UTC] ; stop_time=[Sun 2020-05-24 13:43:42 UTC] ; pid=31481 ; code=exited ; status=0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2020-05-24 13:43:42 UTC", "InactiveEnterTimestampMonotonic": "214828028", "InactiveExitTimestamp": "Sun 2020-05-24 13:43:42 UTC", "InactiveExitTimestampMonotonic": "214829544", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "31484", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", 2020-05-24T13:43:48.323 INFO:teuthology.task.ansible.out: "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2020-05-24 13:43:42 UTC", "WatchdogTimestampMonotonic": "214829523", "WatchdogUSec": "0"}} 2020-05-24T13:43:48.360 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:48 +0000 (0:00:00.419) 0:02:39.587 ************ 2020-05-24T13:43:48.688 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-05-24T13:43:48.690 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004649", "end": "2020-05-24 13:43:48.663245", "rc": 0, "start": "2020-05-24 13:43:48.658596", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-05-24T13:43:48.722 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:48 +0000 (0:00:00.361) 0:02:39.949 ************ 2020-05-24T13:43:49.250 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-05-24T13:43:49.251 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2020-05-24T13:43:49.287 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:49 +0000 (0:00:00.565) 0:02:40.514 ************ 2020-05-24T13:44:11.565 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-05-24T13:44:11.568 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-266.el7.noarch from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-266.el7 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-266.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository\n Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-266.el7 base 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-266.el7 base 497 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydel 2020-05-24T13:44:11.569 INFO:teuthology.task.ansible.out:tarpm not installed.\n--------------------------------------------------------------------------------\nTotal 16 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-266.el7.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-266.el7.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7_7.6.noarch 4/4 \n Verifying : selinux-policy-targeted-3.13.1-266.el7.noarch 1/4 \n Verifying : selinux-policy-3.13.1-266.el7.noarch 2/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 3/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-266.el7 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-266.el7 \n\nComplete!\n"]} 2020-05-24T13:44:11.619 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:11 +0000 (0:00:22.331) 0:03:02.846 ************ 2020-05-24T13:44:12.552 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-05-24T13:44:12.554 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-05-24T13:44:12.597 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:12 +0000 (0:00:00.978) 0:03:03.824 ************ 2020-05-24T13:44:13.880 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-05-24T13:44:13.881 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-266.el7.noarch from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-266.el7 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-266.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository\n Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-266.el7 base 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-266.el7 base 497 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydel 2020-05-24T13:44:13.883 INFO:teuthology.task.ansible.out:tarpm not installed.\n--------------------------------------------------------------------------------\nTotal 14 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-266.el7.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-266.el7.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7_7.6.noarch 4/4 \n Verifying : selinux-policy-targeted-3.13.1-266.el7.noarch 1/4 \n Verifying : selinux-policy-3.13.1-266.el7.noarch 2/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 3/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-266.el7 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-266.el7 \n\nComplete!\n"]} 2020-05-24T13:44:13.925 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:13 +0000 (0:00:01.328) 0:03:05.152 ************ 2020-05-24T13:44:14.523 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-05-24T13:44:14.525 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-05-24T13:44:14.581 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:14 +0000 (0:00:00.655) 0:03:05.807 ************ 2020-05-24T13:44:20.869 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-05-24T13:44:20.871 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.924832", "end": "2020-05-24 13:44:20.833814", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:44:12.908982", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-05-24T13:44:20.920 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:20 +0000 (0:00:06.339) 0:03:12.147 ************ 2020-05-24T13:44:21.971 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-05-24T13:44:21.973 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-05-24T13:44:22.011 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:22 +0000 (0:00:01.090) 0:03:13.237 ************ 2020-05-24T13:44:22.348 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-05-24T13:44:22.350 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.016492", "end": "2020-05-24 13:44:22.316523", "rc": 0, "start": "2020-05-24 13:44:22.300031", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2020-05-24T13:44:22.392 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:22 +0000 (0:00:00.381) 0:03:13.619 ************ 2020-05-24T13:44:22.728 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-05-24T13:44:22.730 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012003", "end": "2020-05-24 13:44:22.702256", "rc": 0, "start": "2020-05-24 13:44:22.690253", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:22.787 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:22 +0000 (0:00:00.395) 0:03:14.014 ************ 2020-05-24T13:44:23.261 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-05-24T13:44:23.264 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.342616", "end": "2020-05-24 13:44:23.227094", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:44:14.884478", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-05-24T13:44:23.296 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:23 +0000 (0:00:00.507) 0:03:14.522 ************ 2020-05-24T13:44:24.313 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-05-24T13:44:24.355 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:24 +0000 (0:00:01.059) 0:03:15.582 ************ 2020-05-24T13:44:24.708 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.015681", "end": "2020-05-24 13:44:24.672206", "rc": 0, "start": "2020-05-24 13:44:24.656525", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2020-05-24T13:44:24.749 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:24 +0000 (0:00:00.394) 0:03:15.976 ************ 2020-05-24T13:44:25.136 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-05-24T13:44:25.138 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012172", "end": "2020-05-24 13:44:25.111397", "rc": 0, "start": "2020-05-24 13:44:25.099225", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:25.184 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:25 +0000 (0:00:00.434) 0:03:16.410 ************ 2020-05-24T13:44:30.847 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-05-24T13:44:30.849 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.724184", "end": "2020-05-24 13:44:30.806526", "rc": 0, "start": "2020-05-24 13:44:23.082342", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:30.890 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:30 +0000 (0:00:05.705) 0:03:22.116 ************ 2020-05-24T13:44:31.283 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-05-24T13:44:31.285 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-05-24T13:44:31.319 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:31 +0000 (0:00:00.429) 0:03:22.546 ************ 2020-05-24T13:44:31.359 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-05-24T13:44:31.361 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:44:31.394 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:31 +0000 (0:00:00.074) 0:03:22.621 ************ 2020-05-24T13:44:31.478 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-05-24T13:44:31.480 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:31.514 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:31 +0000 (0:00:00.118) 0:03:22.740 ************ 2020-05-24T13:44:31.593 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-05-24T13:44:31.595 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "podman-docker", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "python3"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2020-05-24T13:44:31.651 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:31 +0000 (0:00:00.138) 0:03:22.878 ************ 2020-05-24T13:44:31.714 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi193.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:31.749 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:31 +0000 (0:00:00.097) 0:03:22.976 ************ 2020-05-24T13:44:32.082 INFO:teuthology.task.ansible.out: 2020-05-24T13:44:32.084 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:44:32.121 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:32 +0000 (0:00:00.372) 0:03:23.348 ************ 2020-05-24T13:44:32.123 INFO:teuthology.task.ansible.out: 2020-05-24T13:44:32.437 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-05-24T13:44:32.439 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "name": "ubuntu", "state": "present", "system": false} 2020-05-24T13:44:32.481 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:32 +0000 (0:00:00.358) 0:03:23.707 ************ 2020-05-24T13:44:32.520 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:32.577 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:32 +0000 (0:00:00.097) 0:03:23.804 ************ 2020-05-24T13:44:32.634 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:32.681 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:32 +0000 (0:00:00.103) 0:03:23.908 ************ 2020-05-24T13:44:33.027 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-05-24T13:44:33.029 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-05-24T13:44:33.073 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:33 +0000 (0:00:00.391) 0:03:24.300 ************ 2020-05-24T13:44:34.113 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.558121", "end": "2020-05-24 13:44:34.062707", "rc": 0, "start": "2020-05-24 13:44:25.504586", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:34.179 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:34 +0000 (0:00:01.105) 0:03:25.406 ************ 2020-05-24T13:44:34.189 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-05-24T13:44:34.191 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-05-24T13:44:34.243 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:34 +0000 (0:00:00.063) 0:03:25.470 ************ 2020-05-24T13:44:34.315 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-05-24T13:44:34.318 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi193"}, "changed": false} 2020-05-24T13:44:34.358 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:34 +0000 (0:00:00.115) 0:03:25.585 ************ 2020-05-24T13:44:34.609 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-05-24T13:44:34.660 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:34 +0000 (0:00:00.301) 0:03:25.887 ************ 2020-05-24T13:44:34.706 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-05-24T13:44:34.708 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:44:34.746 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:34 +0000 (0:00:00.086) 0:03:25.973 ************ 2020-05-24T13:44:34.822 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:34.873 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:34 +0000 (0:00:00.121) 0:03:26.095 ************ 2020-05-24T13:44:34.954 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [smithi019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "podman-docker", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "python3"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2020-05-24T13:44:35.010 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:35 +0000 (0:00:00.139) 0:03:26.234 ************ 2020-05-24T13:44:35.073 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:35.119 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:35 +0000 (0:00:00.112) 0:03:26.346 ************ 2020-05-24T13:44:35.477 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:44:35.520 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:35 +0000 (0:00:00.397) 0:03:26.744 ************ 2020-05-24T13:44:35.580 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-05-24T13:44:35.582 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi193.front.sepia.ceph.com", "ansible_hostname": "smithi193", "ansible_nodename": "smithi193"}, "changed": true, "name": "smithi193"} 2020-05-24T13:44:35.625 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:35 +0000 (0:00:00.107) 0:03:26.852 ************ 2020-05-24T13:44:35.874 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-05-24T13:44:35.876 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "name": "ubuntu", "state": "present", "system": false} 2020-05-24T13:44:35.919 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:35 +0000 (0:00:00.293) 0:03:27.146 ************ 2020-05-24T13:44:35.962 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-05-24T13:44:35.964 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:35.998 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:35 +0000 (0:00:00.078) 0:03:27.225 ************ 2020-05-24T13:44:36.040 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-05-24T13:44:36.042 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:36.095 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:36 +0000 (0:00:00.093) 0:03:27.318 ************ 2020-05-24T13:44:36.451 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-05-24T13:44:36.453 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-05-24T13:44:36.494 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:36 +0000 (0:00:00.402) 0:03:27.720 ************ 2020-05-24T13:44:36.591 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-05-24T13:44:36.593 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2020-05-24T13:44:36.635 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:36 +0000 (0:00:00.141) 0:03:27.862 ************ 2020-05-24T13:44:37.572 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-05-24T13:44:37.574 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-05-24T13:44:37.623 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:37 +0000 (0:00:00.986) 0:03:28.848 ************ 2020-05-24T13:44:37.662 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi019"}, "changed": false} TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-05-24T13:44:37.704 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:37 +0000 (0:00:00.082) 0:03:28.931 ************ 2020-05-24T13:44:37.753 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:37 +0000 (0:00:00.048) 0:03:28.980 ************ 2020-05-24T13:44:38.276 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-05-24T13:44:38.278 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi019.front.sepia.ceph.com", "ansible_hostname": "smithi019", "ansible_nodename": "smithi019"}, "changed": true, "name": "smithi019"} 2020-05-24T13:44:38.323 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:38 +0000 (0:00:00.570) 0:03:29.550 ************ 2020-05-24T13:44:38.662 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-05-24T13:44:38.664 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-05-24T13:44:38.710 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:38 +0000 (0:00:00.386) 0:03:29.936 ************ 2020-05-24T13:44:39.275 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-05-24T13:44:39.277 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2020-05-24T13:44:39.320 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:39 +0000 (0:00:00.608) 0:03:30.545 ************ 2020-05-24T13:44:39.732 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-05-24T13:44:39.734 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-05-24T13:44:39.791 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:39 +0000 (0:00:00.471) 0:03:31.016 ************ 2020-05-24T13:44:40.125 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-24T13:44:40.128 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:44:40.160 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:40 +0000 (0:00:00.370) 0:03:31.386 ************ 2020-05-24T13:44:40.340 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-05-24T13:44:40.346 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-05-24T13:44:40.402 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:40 +0000 (0:00:00.240) 0:03:31.627 ************ 2020-05-24T13:44:40.530 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "kvm", "state": "present", "system": false} 2020-05-24T13:44:40.565 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:40 +0000 (0:00:00.164) 0:03:31.792 ************ 2020-05-24T13:44:40.948 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-24T13:44:40.950 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1209, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-05-24T13:44:40.982 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:40 +0000 (0:00:00.416) 0:03:32.209 ************ 2020-05-24T13:44:41.306 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-05-24T13:44:41.308 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-05-24T13:44:41.348 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:41 +0000 (0:00:00.361) 0:03:32.570 ************ 2020-05-24T13:44:41.949 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-05-24T13:44:41.988 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:41 +0000 (0:00:00.645) 0:03:33.215 ************ 2020-05-24T13:44:42.361 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-05-24T13:44:42.420 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:42 +0000 (0:00:00.425) 0:03:33.641 ************ 2020-05-24T13:44:42.762 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-24T13:44:42.764 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:44:42.798 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:42 +0000 (0:00:00.382) 0:03:34.023 ************ 2020-05-24T13:44:42.978 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-05-24T13:44:43.015 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:43 +0000 (0:00:00.217) 0:03:34.240 ************ 2020-05-24T13:44:43.149 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-24T13:44:43.150 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "kvm", "state": "present", "system": false} 2020-05-24T13:44:43.183 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:43 +0000 (0:00:00.169) 0:03:34.410 ************ 2020-05-24T13:44:43.609 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-24T13:44:43.610 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1209, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-05-24T13:44:43.649 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:43 +0000 (0:00:00.466) 0:03:34.876 ************ 2020-05-24T13:44:44.697 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-05-24T13:44:44.734 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:44 +0000 (0:00:01.084) 0:03:35.961 ************ 2020-05-24T13:44:45.357 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.000953", "end": "2020-05-24 13:44:45.319182", "rc": 0, "start": "2020-05-24 13:44:43.318229", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:45.411 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:45 +0000 (0:00:00.676) 0:03:36.637 ************ 2020-05-24T13:44:45.453 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-05-24T13:44:45.455 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:45.491 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:45 +0000 (0:00:00.079) 0:03:36.717 ************ 2020-05-24T13:44:45.532 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-05-24T13:44:45.592 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:45 +0000 (0:00:00.101) 0:03:36.819 ************ 2020-05-24T13:44:45.788 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-05-24T13:44:45.838 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:45 +0000 (0:00:00.246) 0:03:37.065 ************ 2020-05-24T13:44:46.663 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-05-24T13:44:46.665 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'key': 'centos7-fcgi-ceph', 'value': {'name': 'CentOS 7 Local fastcgi Repo', 'baseurl': 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2020-05-24T13:44:47.634 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2020-05-24T13:44:47.676 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:47 +0000 (0:00:01.837) 0:03:38.903 ************ 2020-05-24T13:44:47.721 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-05-24T13:44:47.763 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:47 +0000 (0:00:00.087) 0:03:38.990 ************ 2020-05-24T13:44:47.804 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:47.847 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:47 +0000 (0:00:00.083) 0:03:39.074 ************ 2020-05-24T13:44:47.908 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-05-24T13:44:47.910 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:44:47.954 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:47 +0000 (0:00:00.105) 0:03:39.179 ************ 2020-05-24T13:44:47.996 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:48.035 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:48 +0000 (0:00:00.082) 0:03:39.261 ************ 2020-05-24T13:44:48.090 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:44:48.092 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-05-24T13:44:48.145 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:48 +0000 (0:00:00.110) 0:03:39.372 ************ 2020-05-24T13:44:48.283 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-24T13:44:48.286 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.109394", "end": "2020-05-24 13:44:48.250705", "rc": 0, "start": "2020-05-24 13:44:46.141311", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:48.324 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:48 +0000 (0:00:00.179) 0:03:39.551 ************ 2020-05-24T13:44:48.369 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-05-24T13:44:48.371 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:48.408 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:48 +0000 (0:00:00.083) 0:03:39.635 ************ 2020-05-24T13:44:48.437 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-05-24T13:44:48.502 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:48 +0000 (0:00:00.094) 0:03:39.729 ************ 2020-05-24T13:44:48.619 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-05-24T13:44:48.621 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2020-05-24T13:44:48.681 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:48 +0000 (0:00:00.179) 0:03:39.908 ************ 2020-05-24T13:44:49.297 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-05-24T13:44:49.301 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.241068", "end": "2020-05-24 13:44:49.254799", "rc": 0, "start": "2020-05-24 13:44:49.013731", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirrors.mit.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.facebook.net", " * updates: mirror.vcu.edu", "No unfinished transactions left."]} 2020-05-24T13:44:49.341 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:49 +0000 (0:00:00.659) 0:03:40.567 ************ 2020-05-24T13:44:49.537 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** ok: [smithi019.front.sepia.ceph.com] => (item={'key': 'centos7-fcgi-ceph', 'value': {'name': 'CentOS 7 Local fastcgi Repo', 'baseurl': 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2020-05-24T13:44:49.721 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-05-24T13:44:49.722 INFO:teuthology.task.ansible.out:fatal: [smithi193.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.044950", "end": "2020-05-24 13:44:49.682734", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:44:49.637784", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-05-24T13:44:49.777 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:49 +0000 (0:00:00.436) 0:03:41.004 ************ 2020-05-24T13:44:49.846 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:49.893 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:49 +0000 (0:00:00.116) 0:03:41.120 ************ 2020-05-24T13:44:50.584 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-05-24T13:44:50.586 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2020-05-24T13:44:50.624 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:50 +0000 (0:00:00.731) 0:03:41.851 ************ 2020-05-24T13:44:50.671 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-05-24T13:44:50.744 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:50 +0000 (0:00:00.112) 0:03:41.964 ************ 2020-05-24T13:44:50.778 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-05-24T13:44:50.785 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:50.815 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:50 +0000 (0:00:00.077) 0:03:42.042 ************ 2020-05-24T13:44:50.902 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-05-24T13:44:50.903 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:44:50.942 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:50 +0000 (0:00:00.127) 0:03:42.169 ************ 2020-05-24T13:44:50.996 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-05-24T13:44:50.998 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:51.035 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:51 +0000 (0:00:00.092) 0:03:42.261 ************ 2020-05-24T13:44:51.085 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-05-24T13:44:51.129 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:51 +0000 (0:00:00.094) 0:03:42.356 ************ 2020-05-24T13:44:51.636 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-05-24T13:44:51.638 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2020-05-24T13:44:51.670 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:51 +0000 (0:00:00.541) 0:03:42.897 ************ 2020-05-24T13:44:52.239 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-05-24T13:44:52.241 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.203463", "end": "2020-05-24 13:44:52.200958", "rc": 0, "start": "2020-05-24 13:44:51.997495", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirrors.mit.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.facebook.net", " * updates: mirror.vcu.edu", "No unfinished transactions left."]} 2020-05-24T13:44:52.282 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:52 +0000 (0:00:00.610) 0:03:43.508 ************ 2020-05-24T13:44:52.663 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-05-24T13:44:52.664 INFO:teuthology.task.ansible.out:fatal: [smithi019.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.033222", "end": "2020-05-24 13:44:52.629143", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:44:52.595921", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-05-24T13:44:52.722 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:52 +0000 (0:00:00.440) 0:03:43.949 ************ 2020-05-24T13:44:52.766 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-05-24T13:44:52.768 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:52.804 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:52 +0000 (0:00:00.079) 0:03:44.028 ************ 2020-05-24T13:44:58.127 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.4.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7. 2020-05-24T13:44:58.129 INFO:teuthology.task.ansible.out:centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.4 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.4 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.4 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages (+3 Dependent packages)\n\nInstalled size: 22 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 1/6 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 2/6 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 3/6 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/6 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 6/6 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/6 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 2/6 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 3/6 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 4/6 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 5/6 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 6/6 \n\nRemoved:\n librados2.x86_64 1:10.2.5-4.el7 librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.4 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 \n 2020-05-24T13:44:58.132 INFO:teuthology.task.ansible.out: qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 \n\nComplete!\n"]} 2020-05-24T13:44:58.158 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:05.355) 0:03:49.384 ************ 2020-05-24T13:45:01.106 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.4.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img 2020-05-24T13:45:01.108 INFO:teuthology.task.ansible.out:x86_64 10:1.5.3-167.el7_7.4 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.4 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.4 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages (+3 Dependent packages)\n\nInstalled size: 22 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 1/6 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 2/6 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 3/6 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/6 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 6/6 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/6 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 2/6 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 3/6 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 4/6 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 5/6 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 6/6 \n\nRemoved:\n librados2.x86_64 1:10.2.5-4.el7 librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.4 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 \n\n 2020-05-24T13:45:01.110 INFO:teuthology.task.ansible.out:Complete!\n"]} 2020-05-24T13:45:01.174 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:03.008) 0:03:52.392 ************ 2020-05-24T13:45:01.561 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2020-05-24T13:45:01.652 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.486) 0:03:52.878 ************ 2020-05-24T13:45:04.589 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2020-05-24T13:45:04.634 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:02.978) 0:03:55.857 ************ 2020-05-24T13:45:18.644 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-24T13:45:18.647 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-21.el7_7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose 2020-05-24T13:45:18.649 INFO:teuthology.task.ansible.out:is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "podman-docker-1.4.4-4.el7.centos.noarch providing podman-docker is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "iozone-3.424-2_ceph.el7.centos.x86_64 providing iozone is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch provid 2020-05-24T13:45:18.651 INFO:teuthology.task.ansible.out:ing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "python3-3.6.8-10.el7.x86_64 providing python3 is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-173.el7_8.3 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n============================================= 2020-05-24T13:45:18.653 INFO:teuthology.task.ansible.out:===================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-173.el7_8.3 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-28.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-173.el7_8.3 updates 703 k\n qemu-kvm-common x86_64 10:1.5.3-173.el7_8.3 updates 438 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages (+5 Dependent packages)\n\nTotal download size: 7.3 M\nInstalled size: 22 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 16 MB/s | 7.3 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-28.el7.x86_64 1/7 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/7 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/7 \n Installing : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 4/7 \n Installing : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 5/7 \n Installing : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 6/7 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : boost-random-1.53.0-28.el7.x 2020-05-24T13:45:18.655 INFO:teuthology.task.ansible.out:86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 5/7 \n Verifying : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 6/7 \n Verifying : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 7/7 \n\nInstalled:\n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-28.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-173.el7_8.3 \n qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 \n\nComplete!\n"]} 2020-05-24T13:45:18.699 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:18 +0000 (0:00:14.068) 0:04:09.925 ************ 2020-05-24T13:45:19.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-05-24T13:45:19.522 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2020-05-24T13:45:19.561 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.861) 0:04:10.787 ************ 2020-05-24T13:45:20.112 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2020-05-24T13:45:20.154 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:20 +0000 (0:00:00.593) 0:04:11.380 ************ 2020-05-24T13:45:20.195 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:20.237 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:20 +0000 (0:00:00.082) 0:04:11.463 ************ 2020-05-24T13:45:20.672 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-05-24T13:45:20.674 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice basic.target polkit.service dbus.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ip6tables.service iptables.service ebtables.service ipset.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2020-05-24T13:45:20.676 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2020-05-24T13:45:20.678 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:45:20.721 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:20 +0000 (0:00:00.484) 0:04:11.948 ************ 2020-05-24T13:45:21.152 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi193.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-05-24T13:45:21.191 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:21 +0000 (0:00:00.470) 0:04:12.418 ************ 2020-05-24T13:45:22.033 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-24T13:45:22.039 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-21.el7_7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose 2020-05-24T13:45:22.041 INFO:teuthology.task.ansible.out:is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "podman-docker-1.4.4-4.el7.centos.noarch providing podman-docker is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "iozone-3.424-2_ceph.el7.centos.x86_64 providing iozone is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch provid 2020-05-24T13:45:22.043 INFO:teuthology.task.ansible.out:ing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "python3-3.6.8-10.el7.x86_64 providing python3 is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-173.el7_8.3 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n============================================= 2020-05-24T13:45:22.045 INFO:teuthology.task.ansible.out:===================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-173.el7_8.3 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-28.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-173.el7_8.3 updates 703 k\n qemu-kvm-common x86_64 10:1.5.3-173.el7_8.3 updates 438 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages (+5 Dependent packages)\n\nTotal download size: 7.3 M\nInstalled size: 22 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 11 MB/s | 7.3 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-28.el7.x86_64 1/7 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/7 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/7 \n Installing : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 4/7 \n Installing : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 5/7 \n Installing : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 6/7 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : boost-random-1.53.0-28.el7.x 2020-05-24T13:45:22.047 INFO:teuthology.task.ansible.out:86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 2020-05-24T13:45:22.047 INFO:teuthology.task.ansible.out: 4/7 \n Verifying : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 5/7 \n Verifying : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 6/7 \n Verifying : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 7/7 \n\nInstalled:\n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-28.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-173.el7_8.3 \n qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 \n\nComplete!\n"]} 2020-05-24T13:45:22.083 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.891) 0:04:13.309 ************ 2020-05-24T13:45:22.148 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-05-24T13:45:22.187 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.103) 0:04:13.413 ************ 2020-05-24T13:45:22.232 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-05-24T13:45:22.235 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.269 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.082) 0:04:13.495 ************ 2020-05-24T13:45:22.310 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-05-24T13:45:22.350 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.076) 0:04:13.572 ************ 2020-05-24T13:45:22.384 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-05-24T13:45:22.386 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.424 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.079) 0:04:13.651 ************ 2020-05-24T13:45:22.467 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.504 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.079) 0:04:13.731 ************ 2020-05-24T13:45:22.558 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-05-24T13:45:22.560 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.578 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.625 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.120) 0:04:13.851 ************ 2020-05-24T13:45:22.659 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-05-24T13:45:22.661 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.698 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.073) 0:04:13.925 ************ 2020-05-24T13:45:22.728 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-05-24T13:45:22.765 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.064) 0:04:13.989 ************ 2020-05-24T13:45:22.794 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.828 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.065) 0:04:14.055 ************ 2020-05-24T13:45:22.870 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-05-24T13:45:22.873 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:22.913 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.080) 0:04:14.135 ************ 2020-05-24T13:45:22.918 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-05-24T13:45:22.920 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2020-05-24T13:45:22.960 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.051) 0:04:14.186 ************ 2020-05-24T13:45:22.971 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.008 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.047) 0:04:14.234 ************ 2020-05-24T13:45:23.050 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.090 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.082) 0:04:14.317 ************ 2020-05-24T13:45:23.132 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-05-24T13:45:23.134 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.175 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.085) 0:04:14.402 ************ 2020-05-24T13:45:23.206 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.243 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.067) 0:04:14.470 ************ 2020-05-24T13:45:23.286 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-05-24T13:45:23.288 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.326 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.082) 0:04:14.553 ************ 2020-05-24T13:45:23.378 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.421 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.086) 0:04:14.640 ************ 2020-05-24T13:45:23.454 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.491 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.077) 0:04:14.717 ************ 2020-05-24T13:45:23.512 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-24T13:45:23.514 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2020-05-24T13:45:23.550 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.059) 0:04:14.776 ************ 2020-05-24T13:45:23.559 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-05-24T13:45:23.560 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.601 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.050) 0:04:14.827 ************ 2020-05-24T13:45:23.627 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.672 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.071) 0:04:14.899 ************ 2020-05-24T13:45:23.682 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-05-24T13:45:23.684 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.721 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.048) 0:04:14.948 ************ 2020-05-24T13:45:23.750 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-05-24T13:45:23.752 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.786 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.064) 0:04:15.013 ************ 2020-05-24T13:45:23.829 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-05-24T13:45:23.831 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.868 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.082) 0:04:15.095 ************ 2020-05-24T13:45:23.912 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-05-24T13:45:23.915 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.961 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.092) 0:04:15.188 ************ 2020-05-24T13:45:23.989 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.025 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.063) 0:04:15.251 ************ 2020-05-24T13:45:24.067 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-24T13:45:24.070 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.109 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.084) 0:04:15.336 ************ 2020-05-24T13:45:24.134 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service polkit.service basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target ebtables.service iptables.service ipset.service ip6tables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": " 2020-05-24T13:45:24.136 INFO:teuthology.task.ansible.out:yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPr 2020-05-24T13:45:24.137 INFO:teuthology.task.ansible.out:iority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:45:24.172 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.063) 0:04:15.399 ************ 2020-05-24T13:45:24.183 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-24T13:45:24.185 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.221 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.049) 0:04:15.448 ************ 2020-05-24T13:45:24.254 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-05-24T13:45:24.256 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.295 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.073) 0:04:15.522 ************ 2020-05-24T13:45:24.332 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-24T13:45:24.334 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.374 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.074) 0:04:15.596 ************ 2020-05-24T13:45:24.400 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-24T13:45:24.402 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.435 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.065) 0:04:15.662 ************ 2020-05-24T13:45:24.500 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.535 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.099) 0:04:15.762 ************ 2020-05-24T13:45:24.576 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.611 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.076) 0:04:15.838 ************ 2020-05-24T13:45:24.612 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:24.651 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-05-24T13:45:24.653 INFO:teuthology.task.ansible.out:fatal: [smithi019.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring TASK [testnode : set_fact] ***************************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.686 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.074) 0:04:15.912 ************ 2020-05-24T13:45:24.732 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.046) 0:04:15.959 ************ 2020-05-24T13:45:24.763 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.797 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.065) 0:04:16.024 ************ 2020-05-24T13:45:24.842 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.876 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.078) 0:04:16.103 ************ 2020-05-24T13:45:24.923 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.965 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.088) 0:04:16.192 ************ 2020-05-24T13:45:25.026 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.089 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.123) 0:04:16.316 ************ 2020-05-24T13:45:25.098 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-05-24T13:45:25.100 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-05-24T13:45:25.136 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.047) 0:04:16.363 ************ 2020-05-24T13:45:25.170 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-05-24T13:45:25.173 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.206 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.069) 0:04:16.432 ************ 2020-05-24T13:45:25.250 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-05-24T13:45:25.287 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.081) 0:04:16.514 ************ 2020-05-24T13:45:25.328 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-05-24T13:45:25.330 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.382 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.088) 0:04:16.602 ************ 2020-05-24T13:45:25.417 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-05-24T13:45:25.419 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.452 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.076) 0:04:16.679 ************ 2020-05-24T13:45:25.473 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:25.478 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-05-24T13:45:25.510 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.057) 0:04:16.737 ************ 2020-05-24T13:45:25.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-05-24T13:45:25.522 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.574 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.060) 0:04:16.797 ************ 2020-05-24T13:45:25.611 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.650 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.080) 0:04:16.877 ************ 2020-05-24T13:45:25.681 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-05-24T13:45:25.720 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.065) 0:04:16.942 ************ 2020-05-24T13:45:25.765 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.800 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.084) 0:04:17.027 ************ 2020-05-24T13:45:25.841 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.877 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.076) 0:04:17.104 ************ 2020-05-24T13:45:25.886 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-24T13:45:25.888 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:45:25.928 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.051) 0:04:17.155 ************ 2020-05-24T13:45:25.936 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-05-24T13:45:25.938 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.975 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.047) 0:04:17.202 ************ 2020-05-24T13:45:26.007 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-24T13:45:26.009 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.043 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.067) 0:04:17.270 ************ 2020-05-24T13:45:26.083 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.134 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.091) 0:04:17.361 ************ 2020-05-24T13:45:26.167 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-24T13:45:26.169 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.202 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.067) 0:04:17.429 ************ 2020-05-24T13:45:26.232 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-05-24T13:45:26.234 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.266 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.064) 0:04:17.493 ************ 2020-05-24T13:45:26.275 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:26.277 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-05-24T13:45:26.325 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.048) 0:04:17.542 ************ TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.377 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.062) 0:04:17.604 ************ 2020-05-24T13:45:26.388 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.422 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.043) 0:04:17.648 ************ 2020-05-24T13:45:26.432 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.469 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.046) 0:04:17.695 ************ 2020-05-24T13:45:26.478 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.514 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.046) 0:04:17.741 ************ 2020-05-24T13:45:26.524 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-05-24T13:45:26.526 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.565 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.050) 0:04:17.792 ************ 2020-05-24T13:45:26.577 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.614 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.048) 0:04:17.841 ************ 2020-05-24T13:45:26.630 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.666 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.051) 0:04:17.893 ************ 2020-05-24T13:45:26.677 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:26.679 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.727 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.051) 0:04:17.945 ************ TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.767 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.049) 0:04:17.994 ************ 2020-05-24T13:45:26.777 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-24T13:45:26.779 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.817 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.049) 0:04:18.043 ************ 2020-05-24T13:45:26.826 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-05-24T13:45:26.828 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.867 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.050) 0:04:18.094 ************ 2020-05-24T13:45:26.877 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:26.883 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.926 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.058) 0:04:18.152 ************ 2020-05-24T13:45:26.937 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-05-24T13:45:26.939 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:26.978 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.052) 0:04:18.205 ************ 2020-05-24T13:45:26.989 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:26.993 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.025 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.047) 0:04:18.252 ************ 2020-05-24T13:45:27.037 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-24T13:45:27.039 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.080 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.053) 0:04:18.306 ************ 2020-05-24T13:45:27.103 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.140 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.061) 0:04:18.367 ************ 2020-05-24T13:45:27.150 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-24T13:45:27.152 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.194 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.046) 0:04:18.414 ************ 2020-05-24T13:45:27.196 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:27.198 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.238 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.047) 0:04:18.462 ************ 2020-05-24T13:45:27.243 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-24T13:45:27.245 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.281 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.046) 0:04:18.508 ************ 2020-05-24T13:45:27.296 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.328 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.046) 0:04:18.555 ************ 2020-05-24T13:45:27.338 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-05-24T13:45:27.346 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.387 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.055) 0:04:18.611 ************ 2020-05-24T13:45:27.398 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.433 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.048) 0:04:18.660 ************ 2020-05-24T13:45:27.443 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-24T13:45:27.445 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.482 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.048) 0:04:18.709 ************ 2020-05-24T13:45:27.492 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-05-24T13:45:27.494 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.528 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.045) 0:04:18.754 ************ 2020-05-24T13:45:27.538 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-24T13:45:27.539 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.605 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.077) 0:04:18.832 ************ 2020-05-24T13:45:27.620 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.685 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.077) 0:04:18.910 ************ 2020-05-24T13:45:27.694 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.734 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.049) 0:04:18.960 ************ 2020-05-24T13:45:27.753 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.788 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.055) 0:04:19.015 ************ 2020-05-24T13:45:27.799 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-05-24T13:45:27.801 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.838 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.047) 0:04:19.062 ************ 2020-05-24T13:45:27.844 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.878 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.043) 0:04:19.105 ************ 2020-05-24T13:45:27.890 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.925 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.046) 0:04:19.152 ************ 2020-05-24T13:45:27.935 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:27.977 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.052) 0:04:19.204 ************ 2020-05-24T13:45:27.985 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-24T13:45:27.987 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.021 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.043) 0:04:19.247 ************ 2020-05-24T13:45:28.030 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-05-24T13:45:28.041 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.067 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.046) 0:04:19.294 ************ 2020-05-24T13:45:28.078 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.114 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.047) 0:04:19.341 ************ 2020-05-24T13:45:28.124 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-05-24T13:45:28.126 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.164 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.049) 0:04:19.391 ************ 2020-05-24T13:45:28.178 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.216 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.052) 0:04:19.443 ************ 2020-05-24T13:45:28.226 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-05-24T13:45:28.228 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.270 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.053) 0:04:19.497 ************ 2020-05-24T13:45:28.272 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:28.281 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-05-24T13:45:28.286 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.319 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.049) 0:04:19.546 ************ 2020-05-24T13:45:28.321 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:28.332 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:28.334 INFO:teuthology.task.ansible.out:TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.388 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.068) 0:04:19.615 ************ 2020-05-24T13:45:28.423 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.500 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.105) 0:04:19.720 ************ 2020-05-24T13:45:28.536 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.576 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.081) 0:04:19.802 ************ 2020-05-24T13:45:28.650 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-05-24T13:45:28.680 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.103) 0:04:19.905 ************ 2020-05-24T13:45:28.691 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:28.693 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-05-24T13:45:28.734 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.055) 0:04:19.960 ************ 2020-05-24T13:45:28.758 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-05-24T13:45:28.760 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi193.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi193.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.770 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.785 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.805 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.842 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.107) 0:04:20.068 ************ 2020-05-24T13:45:29.106 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-24T13:45:29.112 INFO:teuthology.task.ansible.out: ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:45:29.164 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.318) 0:04:20.387 ************ 2020-05-24T13:45:29.225 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-05-24T13:45:29.227 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.038741", "end": "2020-05-24 13:45:29.178652", "rc": 0, "start": "2020-05-24 13:45:29.139911", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-05-24T13:45:29.271 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.110) 0:04:20.498 ************ 2020-05-24T13:45:29.322 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.370 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.098) 0:04:20.596 ************ 2020-05-24T13:45:29.433 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.494 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.122) 0:04:20.718 ************ 2020-05-24T13:45:29.534 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-05-24T13:45:29.576 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.084) 0:04:20.802 ************ 2020-05-24T13:45:29.588 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.636 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.059) 0:04:20.862 ************ 2020-05-24T13:45:29.646 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.684 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.048) 0:04:20.911 ************ 2020-05-24T13:45:29.696 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-05-24T13:45:29.698 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.736 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.052) 0:04:20.963 ************ 2020-05-24T13:45:29.747 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.795 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.058) 0:04:21.022 ************ 2020-05-24T13:45:29.850 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-05-24T13:45:29.852 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.906 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.110) 0:04:21.133 ************ 2020-05-24T13:45:29.952 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:29.954 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.992 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.086) 0:04:21.219 ************ 2020-05-24T13:45:30.044 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-24T13:45:30.046 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.092 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.099) 0:04:21.319 ************ 2020-05-24T13:45:30.150 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.181 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.087) 0:04:21.406 ************ 2020-05-24T13:45:30.225 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:30.227 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.229 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-05-24T13:45:30.230 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2020-05-24T13:45:30.283 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.102) 0:04:21.509 ************ 2020-05-24T13:45:30.365 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.082) 0:04:21.592 ************ 2020-05-24T13:45:30.380 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-24T13:45:30.382 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.435 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.069) 0:04:21.661 ************ 2020-05-24T13:45:30.454 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-24T13:45:30.456 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item={'block_used': 13047397, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277087, 'size_available': 930926948352, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91466, 'block_size': 4096, 'inode_available': 60963510}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277087, "block_size": 4096, "block_total": 240324484, "block_used": 13047397, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963510, "inode_total": 61054976, "inode_used": 91466, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926948352, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}} 2020-05-24T13:45:30.513 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.078) 0:04:21.740 ************ 2020-05-24T13:45:30.524 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.578 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.065) 0:04:21.805 ************ 2020-05-24T13:45:30.649 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-05-24T13:45:30.651 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.698 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.119) 0:04:21.924 ************ 2020-05-24T13:45:30.754 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-05-24T13:45:30.756 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.796 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.098) 0:04:22.023 ************ 2020-05-24T13:45:30.841 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-05-24T13:45:30.843 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:30.890 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.093) 0:04:22.117 ************ 2020-05-24T13:45:30.900 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011964", "end": "2020-05-24 13:45:30.861522", "rc": 0, "start": "2020-05-24 13:45:30.849558", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:30.935 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.044) 0:04:22.162 ************ 2020-05-24T13:45:30.954 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-05-24T13:45:30.955 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.013 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.078) 0:04:22.240 ************ 2020-05-24T13:45:31.023 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-05-24T13:45:31.025 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'block_used': 13047397, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277087, 'size_available': 930926948352, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91466, 'block_size': 4096, 'inode_available': 60963510}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277087, "block_size": 4096, "block_total": 240324484, "block_used": 13047397, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963510, "inode_total": 61054976, "inode_used": 91466, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926948352, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.068 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.055) 0:04:22.295 ************ 2020-05-24T13:45:31.078 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.123 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.052) 0:04:22.348 ************ 2020-05-24T13:45:31.134 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-05-24T13:45:31.181 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.059) 0:04:22.408 ************ 2020-05-24T13:45:31.200 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-05-24T13:45:31.201 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.253 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.072) 0:04:22.480 ************ 2020-05-24T13:45:31.270 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi193.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XN91', 'wwn-0x5000c5009293808d'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XN91-part1', 'wwn-0x5000c5009293808d-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293808d', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XN91', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN91", "wwn-0x5000c5009293808d"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN91-part1", "wwn-0x5000c5009293808d-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XN91", "size": "931.51 GB", "suppor 2020-05-24T13:45:31.273 INFO:teuthology.task.ansible.out:t_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293808d"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.303 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.342 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.088) 0:04:22.569 ************ 2020-05-24T13:45:31.389 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.427 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.080) 0:04:22.649 ************ 2020-05-24T13:45:31.467 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.525 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.102) 0:04:22.752 ************ 2020-05-24T13:45:31.575 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.616 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.090) 0:04:22.843 ************ 2020-05-24T13:45:31.683 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-05-24T13:45:31.686 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.722 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.106) 0:04:22.949 ************ 2020-05-24T13:45:31.759 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-05-24T13:45:31.762 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.814 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.088) 0:04:23.038 ************ 2020-05-24T13:45:31.852 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-05-24T13:45:31.854 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.895 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.082) 0:04:23.121 ************ 2020-05-24T13:45:31.943 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:31.950 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-05-24T13:45:32.009 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.115) 0:04:23.236 ************ 2020-05-24T13:45:32.098 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi019.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.128 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-05-24T13:45:32.134 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.142 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi019.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.161 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.218 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.208) 0:04:23.445 ************ 2020-05-24T13:45:32.530 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-05-24T13:45:32.534 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.024251", "end": "2020-05-24 13:45:32.489737", "rc": 0, "start": "2020-05-24 13:45:32.465486", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-05-24T13:45:32.581 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.362) 0:04:23.807 ************ 2020-05-24T13:45:32.627 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-05-24T13:45:32.630 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.702 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.121) 0:04:23.929 ************ 2020-05-24T13:45:32.758 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-05-24T13:45:32.760 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.793 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.091) 0:04:24.020 ************ 2020-05-24T13:45:32.847 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.880 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.084) 0:04:24.105 ************ 2020-05-24T13:45:32.928 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-05-24T13:45:32.930 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.972 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.093) 0:04:24.199 ************ 2020-05-24T13:45:33.077 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ST8MUr-DntL-krxD-79ZU-XMh6-2lTY-H5dYdN', 'nvme-INTEL_SSDPEDMD400G4_PHFT620500HB400BGN', 'nvme-nvme.8086-50484654363230353030484234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPEDMD400G4', 'serial': 'PHFT620500HB400BGN', 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.551501", "end": "2020-05-24 13:45:33.038372", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-ST8MUr-DntL-krxD-79ZU-XMh6-2lTY-H5dYdN", "nvme-INTEL_SSDPEDMD400G4_PHFT620500HB400BGN", "nvme-nvme.8086-50484654363230353030484234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc 2020-05-24T13:45:33.082 INFO:teuthology.task.ansible.out:": 0, "start": "2020-05-24 13:45:31.486871", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:33.502 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2020-05-24T13:45:33.542 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.567) 0:04:24.767 ************ 2020-05-24T13:45:33.604 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-24T13:45:33.607 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item={'block_used': 13047431, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277053, 'size_available': 930926809088, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91470, 'block_size': 4096, 'inode_available': 60963506}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277053, "block_size": 4096, "block_total": 240324484, "block_used": 13047431, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963506, "inode_total": 61054976, "inode_used": 91470, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926809088, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}} 2020-05-24T13:45:33.642 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.101) 0:04:24.869 ************ 2020-05-24T13:45:33.984 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-05-24T13:45:33.986 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.014606", "end": "2020-05-24 13:45:33.939968", "rc": 0, "start": "2020-05-24 13:45:33.925362", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:34.035 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:34 +0000 (0:00:00.393) 0:04:25.262 ************ 2020-05-24T13:45:34.094 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-05-24T13:45:34.096 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'block_used': 13047431, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277053, 'size_available': 930926809088, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91470, 'block_size': 4096, 'inode_available': 60963506}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277053, "block_size": 4096, "block_total": 240324484, "block_used": 13047431, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963506, "inode_total": 61054976, "inode_used": 91470, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926809088, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:34.147 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:34 +0000 (0:00:00.111) 0:04:25.373 ************ 2020-05-24T13:45:34.177 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-05-24T13:45:34.225 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:34 +0000 (0:00:00.078) 0:04:25.451 ************ 2020-05-24T13:45:34.282 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi019.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J32D', 'wwn-0x5000c500871b6a09'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J32D-part1', 'wwn-0x5000c500871b6a09-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b6a09', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J32D', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J32D", "wwn-0x5000c500871b6a09"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J32D-part1", "wwn-0x5000c500871b6a09-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J32D", "size": "931.51 GB", "suppor 2020-05-24T13:45:34.288 INFO:teuthology.task.ansible.out:t_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b6a09"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:34.481 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFa01N0XHsbkueEk42CwHsg8G478QX2Q03C'], 'uuids': ['2b2b2009-77af-4e8e-a300-baa15fdb5ba2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.060552", "end": "2020-05-24 13:45:34.432048", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFa01N0XHsbkueEk42CwHsg8G478QX2Q03C"], "labels": [], "masters": [], "uuids": ["2b2b2009-77af-4e8e-a300-baa15fdb5ba2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:33.371496", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:35.888 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaKwhWDJ8Bx8QTmbBdAJT2fDukRTXsCexh'], 'uuids': ['6415f868-b8bd-4f91-a37f-d004f8cd1066']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.062384", "end": "2020-05-24 13:45:35.840720", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaKwhWDJ8Bx8QTmbBdAJT2fDukRTXsCexh"], "labels": [], "masters": [], "uuids": ["6415f868-b8bd-4f91-a37f-d004f8cd1066"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:34.778336", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:36.219 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-o0aF9V-I6xC-Adof-ekXy-tsNq-vxsC-MscwBO', 'nvme-INTEL_SSDPEDMD400G4_CVFT5330002G400BGN', 'nvme-nvme.8086-43564654353333303030324734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPEDMD400G4', 'serial': 'CVFT5330002G400BGN', 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.565420", "end": "2020-05-24 13:45:36.168646", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-o0aF9V-I6xC-Adof-ekXy-tsNq-vxsC-MscwBO", "nvme-INTEL_SSDPEDMD400G4_CVFT5330002G400BGN", "nvme-nvme.8086-43564654353333303030324734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:34.603226", "stderr": "", "stderr_lines": [], "stdo 2020-05-24T13:45:36.221 INFO:teuthology.task.ansible.out:ut": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:37.247 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaQeQY7zjR3aVThi7qf9IfqmejjovFzc2A'], 'uuids': ['6ae75872-cd6f-446a-a073-8e8a0584a8c0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.047715", "end": "2020-05-24 13:45:37.207532", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaQeQY7zjR3aVThi7qf9IfqmejjovFzc2A"], "labels": [], "masters": [], "uuids": ["6ae75872-cd6f-446a-a073-8e8a0584a8c0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:36.159817", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:37.646 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQXGwE5qqqUW35q5iBjTaDp8VDoXYikAQF'], 'uuids': ['c0bcb162-9d62-4609-bbd5-7b84e801dca4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.084194", "end": "2020-05-24 13:45:37.596367", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQXGwE5qqqUW35q5iBjTaDp8VDoXYikAQF"], "labels": [], "masters": [], "uuids": ["c0bcb162-9d62-4609-bbd5-7b84e801dca4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:36.512173", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:38.620 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFah9oh0XSOyYa7C9kKGj6GtIXfBgTYvmra'], 'uuids': ['bde2ab10-9ecd-4fae-9721-87c223ce0b10']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.051603", "end": "2020-05-24 13:45:38.582462", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFah9oh0XSOyYa7C9kKGj6GtIXfBgTYvmra"], "labels": [], "masters": [], "uuids": ["bde2ab10-9ecd-4fae-9721-87c223ce0b10"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:37.530859", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:39.032 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQvjTSh0JhEnWj4wbvNupzoir116WWfQ1B'], 'uuids': ['1abccf83-04c9-4e8d-9057-07ac0146208a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.049427", "end": "2020-05-24 13:45:38.987741", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQvjTSh0JhEnWj4wbvNupzoir116WWfQ1B"], "labels": [], "masters": [], "uuids": ["1abccf83-04c9-4e8d-9057-07ac0146208a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:37.938314", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:40.012 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFacZMbEbw2FDyKOKnYYHuZC3TeDnb64IKE'], 'uuids': ['afae54b3-dfba-453c-b2f9-e4cad4cd95a9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.051461", "end": "2020-05-24 13:45:39.966786", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFacZMbEbw2FDyKOKnYYHuZC3TeDnb64IKE"], "labels": [], "masters": [], "uuids": ["afae54b3-dfba-453c-b2f9-e4cad4cd95a9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:38.915325", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:40.072 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:05.847) 0:04:31.299 ************ 2020-05-24T13:45:40.129 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:40.131 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XN91', 'wwn-0x5000c5009293808d'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XN91-part1', 'wwn-0x5000c5009293808d-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293808d', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XN91', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN91", "wwn-0x5000c5009293808d"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN91-part1", "wwn-0x5000c5009293808d-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XN91", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293808d"}}, "skip 2020-05-24T13:45:40.133 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} 2020-05-24T13:45:40.436 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-05-24T13:45:40.438 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQ4VhuUpYrfyTKiV5hPXQX0l9rCuS8N7a0'], 'uuids': ['c82c0517-9139-45bf-b31e-04a3c7c4ea63']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.043276", "end": "2020-05-24 13:45:40.371448", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQ4VhuUpYrfyTKiV5hPXQX0l9rCuS8N7a0"], "labels": [], "masters": [], "uuids": ["c82c0517-9139-45bf-b31e-04a3c7c4ea63"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:39.328172", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:40.510 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:40.512 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ST8MUr-DntL-krxD-79ZU-XMh6-2lTY-H5dYdN', 'nvme-INTEL_SSDPEDMD400G4_PHFT620500HB400BGN', 'nvme-nvme.8086-50484654363230353030484234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPEDMD400G4', 'serial': 'PHFT620500HB400BGN', 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.060217", "end": "2020-05-24 13:45:40.467976", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-ST8MUr-DntL-krxD-79ZU-XMh6-2lTY-H5dYdN", "nvme-INTEL_SSDPEDMD400G4_PHFT620500HB400BGN", "nvme-nvme.8086-50484654363230353030484234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:40.407759", "stderr": "110+0 records in\n110+0 records out\n115343 2020-05-24T13:45:40.514 INFO:teuthology.task.ansible.out:360 bytes (115 MB) copied, 0.0542101 s, 2.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0542101 s, 2.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:40.967 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFa01N0XHsbkueEk42CwHsg8G478QX2Q03C'], 'uuids': ['2b2b2009-77af-4e8e-a300-baa15fdb5ba2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.203682", "end": "2020-05-24 13:45:40.930402", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFa01N0XHsbkueEk42CwHsg8G478QX2Q03C"], "labels": [], "masters": [], "uuids": ["2b2b2009-77af-4e8e-a300-baa15fdb5ba2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:40.726720", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.19746 s, 584 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.19746 s, 584 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:41.525 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaKwhWDJ8Bx8QTmbBdAJT2fDukRTXsCexh'], 'uuids': ['6415f868-b8bd-4f91-a37f-d004f8cd1066']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.209216", "end": "2020-05-24 13:45:41.475844", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaKwhWDJ8Bx8QTmbBdAJT2fDukRTXsCexh"], "labels": [], "masters": [], "uuids": ["6415f868-b8bd-4f91-a37f-d004f8cd1066"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:41.266628", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.202589 s, 569 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.202589 s, 569 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:41.800 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQb9FVSDwCDbI6gNjqmryxR3g1UI60PCAA'], 'uuids': ['c0d30a76-f15f-4f4b-900a-782c832f4511']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.055452", "end": "2020-05-24 13:45:41.763386", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQb9FVSDwCDbI6gNjqmryxR3g1UI60PCAA"], "labels": [], "masters": [], "uuids": ["c0d30a76-f15f-4f4b-900a-782c832f4511"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:40.707934", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other uti 2020-05-24T13:45:41.802 INFO:teuthology.task.ansible.out:lities."]} 2020-05-24T13:45:42.001 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaQeQY7zjR3aVThi7qf9IfqmejjovFzc2A'], 'uuids': ['6ae75872-cd6f-446a-a073-8e8a0584a8c0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.171759", "end": "2020-05-24 13:45:41.967234", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFaQeQY7zjR3aVThi7qf9IfqmejjovFzc2A"], "labels": [], "masters": [], "uuids": ["6ae75872-cd6f-446a-a073-8e8a0584a8c0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:41.795475", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.165162 s, 698 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.165162 s, 698 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:42.473 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFah9oh0XSOyYa7C9kKGj6GtIXfBgTYvmra'], 'uuids': ['bde2ab10-9ecd-4fae-9721-87c223ce0b10']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.207640", "end": "2020-05-24 13:45:42.432542", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFah9oh0XSOyYa7C9kKGj6GtIXfBgTYvmra"], "labels": [], "masters": [], "uuids": ["bde2ab10-9ecd-4fae-9721-87c223ce0b10"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:42.224902", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.201121 s, 574 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.201121 s, 574 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:43.002 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFacZMbEbw2FDyKOKnYYHuZC3TeDnb64IKE'], 'uuids': ['afae54b3-dfba-453c-b2f9-e4cad4cd95a9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHFT620500HB400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.204745", "end": "2020-05-24 13:45:42.957449", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-5QxROprGEoFHkbc9XtDaToWmJIa3uLFacZMbEbw2FDyKOKnYYHuZC3TeDnb64IKE"], "labels": [], "masters": [], "uuids": ["afae54b3-dfba-453c-b2f9-e4cad4cd95a9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620500HB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:42.752704", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.198283 s, 582 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.198283 s, 582 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:43.039 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:43 +0000 (0:00:02.967) 0:04:34.266 ************ 2020-05-24T13:45:43.213 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQpdoGZF0LAd5psDUvZQcPM0EW32ZxOk6q'], 'uuids': ['145a89d4-28ca-40b7-958a-99254068dae9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.048087", "end": "2020-05-24 13:45:43.130546", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQpdoGZF0LAd5psDUvZQcPM0EW32ZxOk6q"], "labels": [], "masters": [], "uuids": ["145a89d4-28ca-40b7-958a-99254068dae9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:42.082459", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other u 2020-05-24T13:45:43.215 INFO:teuthology.task.ansible.out:tilities."]} 2020-05-24T13:45:43.235 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:43 +0000 (0:00:00.195) 0:04:34.462 ************ 2020-05-24T13:45:43.298 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:43.302 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J32D', 'wwn-0x5000c500871b6a09'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J32D-part1', 'wwn-0x5000c500871b6a09-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b6a09', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J32D', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J32D", "wwn-0x5000c500871b6a09"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J32D-part1", "wwn-0x5000c500871b6a09-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J32D", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b6a09"}}, "skip 2020-05-24T13:45:43.303 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} 2020-05-24T13:45:43.479 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-05-24T13:45:43.481 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.084161", "end": "2020-05-24 13:45:43.435529", "rc": 0, "start": "2020-05-24 13:45:43.351368", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:43.516 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:43 +0000 (0:00:00.281) 0:04:34.743 ************ 2020-05-24T13:45:43.725 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:43.727 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-o0aF9V-I6xC-Adof-ekXy-tsNq-vxsC-MscwBO', 'nvme-INTEL_SSDPEDMD400G4_CVFT5330002G400BGN', 'nvme-nvme.8086-43564654353333303030324734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPEDMD400G4', 'serial': 'CVFT5330002G400BGN', 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.086919", "end": "2020-05-24 13:45:43.686186", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-o0aF9V-I6xC-Adof-ekXy-tsNq-vxsC-MscwBO", "nvme-INTEL_SSDPEDMD400G4_CVFT5330002G400BGN", "nvme-nvme.8086-43564654353333303030324734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:43.599267", "stderr": "110+0 records in\n110+0 records out\n115343 2020-05-24T13:45:43.729 INFO:teuthology.task.ansible.out:360 bytes (115 MB) copied, 0.0804468 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0804468 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:43.890 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.049822", "end": "2020-05-24 13:45:43.855645", "rc": 0, "start": "2020-05-24 13:45:43.805823", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:43.928 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:43 +0000 (0:00:00.411) 0:04:35.155 ************ 2020-05-24T13:45:43.957 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-05-24T13:45:44.006 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.077) 0:04:35.232 ************ 2020-05-24T13:45:44.048 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-05-24T13:45:44.052 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:44.098 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.092) 0:04:35.325 ************ 2020-05-24T13:45:44.138 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-05-24T13:45:44.140 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:44.181 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.083) 0:04:35.408 ************ 2020-05-24T13:45:44.217 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-24T13:45:44.219 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:44.266 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.079) 0:04:35.487 ************ 2020-05-24T13:45:44.274 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQXGwE5qqqUW35q5iBjTaDp8VDoXYikAQF'], 'uuids': ['c0bcb162-9d62-4609-bbd5-7b84e801dca4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.189495", "end": "2020-05-24 13:45:44.209628", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQXGwE5qqqUW35q5iBjTaDp8VDoXYikAQF"], "labels": [], "masters": [], "uuids": ["c0bcb162-9d62-4609-bbd5-7b84e801dca4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:44.020133", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.183028 s, 630 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.183028 s, 630 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:44.295 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-05-24T13:45:44.298 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:44.334 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.073) 0:04:35.561 ************ 2020-05-24T13:45:44.678 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi193.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:45:44.731 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.391) 0:04:35.953 ************ 2020-05-24T13:45:44.748 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:44.750 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQvjTSh0JhEnWj4wbvNupzoir116WWfQ1B'], 'uuids': ['1abccf83-04c9-4e8d-9057-07ac0146208a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.166070", "end": "2020-05-24 13:45:44.708169", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQvjTSh0JhEnWj4wbvNupzoir116WWfQ1B"], "labels": [], "masters": [], "uuids": ["1abccf83-04c9-4e8d-9057-07ac0146208a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:44.542099", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.159991 s, 721 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.159991 s, 721 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:44.785 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-24T13:45:44.787 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'block_used': 13047397, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277087, 'size_available': 930926948352, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91466, 'block_size': 4096, 'inode_available': 60963510}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277087, "block_size": 4096, "block_total": 240324484, "block_used": 13047397, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963510, "inode_total": 61054976, "inode_used": 91466, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926948352, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:44.829 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.102) 0:04:36.055 ************ 2020-05-24T13:45:44.871 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-05-24T13:45:44.873 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:44.922 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:44 +0000 (0:00:00.093) 0:04:36.149 ************ 2020-05-24T13:45:44.973 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:45.003 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:45 +0000 (0:00:00.080) 0:04:36.230 ************ 2020-05-24T13:45:45.043 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:45.085 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:45 +0000 (0:00:00.075) 0:04:36.306 ************ 2020-05-24T13:45:45.118 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-05-24T13:45:45.119 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:45.153 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:45 +0000 (0:00:00.073) 0:04:36.379 ************ 2020-05-24T13:45:45.194 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:45.235 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:45 +0000 (0:00:00.081) 0:04:36.460 ************ 2020-05-24T13:45:45.244 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:45.249 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQ4VhuUpYrfyTKiV5hPXQX0l9rCuS8N7a0'], 'uuids': ['c82c0517-9139-45bf-b31e-04a3c7c4ea63']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.180080", "end": "2020-05-24 13:45:45.202549", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQ4VhuUpYrfyTKiV5hPXQX0l9rCuS8N7a0"], "labels": [], "masters": [], "uuids": ["c82c0517-9139-45bf-b31e-04a3c7c4ea63"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:45.022469", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.173435 s, 665 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.173435 s, 665 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:45.269 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-05-24T13:45:45.272 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:45.321 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:45 +0000 (0:00:00.087) 0:04:36.548 ************ 2020-05-24T13:45:45.729 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:45.731 INFO:teuthology.task.ansible.out:TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQb9FVSDwCDbI6gNjqmryxR3g1UI60PCAA'], 'uuids': ['c0d30a76-f15f-4f4b-900a-782c832f4511']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.182824", "end": "2020-05-24 13:45:45.694017", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQb9FVSDwCDbI6gNjqmryxR3g1UI60PCAA"], "labels": [], "masters": [], "uuids": ["c0d30a76-f15f-4f4b-900a-782c832f4511"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:45.511193", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.177056 s, 651 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.177056 s, 651 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:46.118 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-05-24T13:45:46.120 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-05-24T13:45:46.179 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:46 +0000 (0:00:00.858) 0:04:37.406 ************ 2020-05-24T13:45:46.212 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:46.214 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQpdoGZF0LAd5psDUvZQcPM0EW32ZxOk6q'], 'uuids': ['145a89d4-28ca-40b7-958a-99254068dae9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT5330002G400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.175095", "end": "2020-05-24 13:45:46.170227", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-gwjxIq74Mj3LiQmNS1rMTHZwEilJQ3WQpdoGZF0LAd5psDUvZQcPM0EW32ZxOk6q"], "labels": [], "masters": [], "uuids": ["145a89d4-28ca-40b7-958a-99254068dae9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330002G400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:45.995132", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.16886 s, 683 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.16886 s, 683 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:46.284 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:46 +0000 (0:00:00.104) 0:04:37.511 ************ 2020-05-24T13:45:46.707 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-05-24T13:45:46.709 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.093876", "end": "2020-05-24 13:45:46.672233", "rc": 0, "start": "2020-05-24 13:45:46.578357", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:46.754 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:46 +0000 (0:00:00.470) 0:04:37.981 ************ 2020-05-24T13:45:47.012 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:47.014 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:47.137 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.052278", "end": "2020-05-24 13:45:47.103430", "rc": 0, "start": "2020-05-24 13:45:47.051152", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:47.189 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:47 +0000 (0:00:00.434) 0:04:38.416 ************ 2020-05-24T13:45:47.224 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-05-24T13:45:47.268 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:47 +0000 (0:00:00.079) 0:04:38.495 ************ 2020-05-24T13:45:47.311 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-05-24T13:45:47.313 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:47.357 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:47 +0000 (0:00:00.088) 0:04:38.584 ************ 2020-05-24T13:45:47.389 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-05-24T13:45:47.390 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:47.423 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:47 +0000 (0:00:00.066) 0:04:38.650 ************ 2020-05-24T13:45:47.466 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-24T13:45:47.468 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:47.512 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:47 +0000 (0:00:00.088) 0:04:38.739 ************ 2020-05-24T13:45:47.525 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:47.527 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:47.553 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-05-24T13:45:47.555 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:47.602 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:47 +0000 (0:00:00.089) 0:04:38.829 ************ 2020-05-24T13:45:47.963 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:45:47.994 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:47 +0000 (0:00:00.392) 0:04:39.221 ************ 2020-05-24T13:45:48.003 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:48.005 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:48.048 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi019.front.sepia.ceph.com] => (item={'block_used': 13047431, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277053, 'size_available': 930926809088, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91470, 'block_size': 4096, 'inode_available': 60963506}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277053, "block_size": 4096, "block_total": 240324484, "block_used": 13047431, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963506, "inode_total": 61054976, "inode_used": 91470, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926809088, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:48.082 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.087) 0:04:39.309 ************ 2020-05-24T13:45:48.132 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-05-24T13:45:48.134 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:48.166 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.083) 0:04:39.393 ************ 2020-05-24T13:45:48.212 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:48.239 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.073) 0:04:39.466 ************ 2020-05-24T13:45:48.284 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:48.286 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:48.338 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.099) 0:04:39.565 ************ 2020-05-24T13:45:48.390 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-05-24T13:45:48.392 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:48.427 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.089) 0:04:39.654 ************ 2020-05-24T13:45:48.446 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:48.448 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:48.478 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-05-24T13:45:48.480 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:48.522 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.094) 0:04:39.749 ************ 2020-05-24T13:45:48.571 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-05-24T13:45:48.573 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:48.610 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.087) 0:04:39.836 ************ 2020-05-24T13:45:48.894 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:48.934 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:48 +0000 (0:00:00.324) 0:04:40.160 ************ 2020-05-24T13:45:49.178 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-05-24T13:45:49.180 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-05-24T13:45:49.237 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:49 +0000 (0:00:00.302) 0:04:40.463 ************ 2020-05-24T13:45:49.249 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-24T13:45:49.251 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-05-24T13:45:49.300 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:49 +0000 (0:00:00.062) 0:04:40.526 ************ 2020-05-24T13:45:49.703 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:49.705 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:49.815 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:49.817 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:50.042 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:50.044 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:50.298 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:50.300 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:50.355 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:50.357 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:50.683 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:50.694 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:50.732 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:50 +0000 (0:00:01.432) 0:04:41.959 ************ 2020-05-24T13:45:50.783 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:50.785 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:51.053 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-05-24T13:45:51.055 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:45:51.095 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:51 +0000 (0:00:00.363) 0:04:42.322 ************ 2020-05-24T13:45:51.168 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-05-24T13:45:51.170 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-05-24T13:45:51.219 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:51 +0000 (0:00:00.118) 0:04:42.440 ************ 2020-05-24T13:45:51.246 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:51.247 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:51.725 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:51.762 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:51 +0000 (0:00:00.547) 0:04:42.988 ************ 2020-05-24T13:45:51.937 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-05-24T13:45:51.939 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true} 2020-05-24T13:45:51.974 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:51 +0000 (0:00:00.212) 0:04:43.201 ************ 2020-05-24T13:45:52.110 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-24T13:45:52.112 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-05-24T13:45:52.148 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:52 +0000 (0:00:00.172) 0:04:43.374 ************ 2020-05-24T13:45:52.506 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:52.508 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:52.695 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-05-24T13:45:52.758 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:52 +0000 (0:00:00.609) 0:04:43.983 ************ 2020-05-24T13:45:52.786 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-05-24T13:45:52.788 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:52.826 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:52 +0000 (0:00:00.070) 0:04:44.053 ************ 2020-05-24T13:45:52.851 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:52.852 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:52.862 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-05-24T13:45:52.864 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:52.896 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:52 +0000 (0:00:00.069) 0:04:44.123 ************ 2020-05-24T13:45:52.942 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-05-24T13:45:52.944 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:52.987 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:52 +0000 (0:00:00.090) 0:04:44.214 ************ 2020-05-24T13:45:53.027 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:53.062 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.075) 0:04:44.289 ************ 2020-05-24T13:45:53.094 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:53.131 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.068) 0:04:44.358 ************ 2020-05-24T13:45:53.173 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-05-24T13:45:53.174 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:53.176 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:53.218 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.081) 0:04:44.440 ************ 2020-05-24T13:45:53.242 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-05-24T13:45:53.244 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:53.285 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.071) 0:04:44.512 ************ 2020-05-24T13:45:53.511 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:53.513 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:53.524 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:53.562 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.276) 0:04:44.789 ************ 2020-05-24T13:45:53.759 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-05-24T13:45:53.760 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2020-05-24T13:45:53.795 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.233) 0:04:45.022 ************ 2020-05-24T13:45:53.828 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-05-24T13:45:53.830 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:53.864 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.068) 0:04:45.091 ************ 2020-05-24T13:45:53.903 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-05-24T13:45:53.905 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:53.940 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.076) 0:04:45.167 ************ 2020-05-24T13:45:53.949 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-05-24T13:45:53.951 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:45:53.991 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:53 +0000 (0:00:00.050) 0:04:45.218 ************ 2020-05-24T13:45:54.048 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-05-24T13:45:54.050 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-05-24T13:45:54.102 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:54 +0000 (0:00:00.111) 0:04:45.329 ************ 2020-05-24T13:45:54.538 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-05-24T13:45:54.540 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true} 2020-05-24T13:45:54.599 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:54 +0000 (0:00:00.495) 0:04:45.826 ************ 2020-05-24T13:45:54.813 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-05-24T13:45:54.817 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2020-05-24T13:45:54.867 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:54 +0000 (0:00:00.267) 0:04:46.094 ************ 2020-05-24T13:45:54.909 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:54.942 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:54 +0000 (0:00:00.074) 0:04:46.169 ************ 2020-05-24T13:45:55.081 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-05-24T13:45:55.142 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.199) 0:04:46.369 ************ 2020-05-24T13:45:55.172 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-05-24T13:45:55.174 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.208 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.066) 0:04:46.435 ************ 2020-05-24T13:45:55.248 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-05-24T13:45:55.250 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.286 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.077) 0:04:46.512 ************ 2020-05-24T13:45:55.325 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-05-24T13:45:55.327 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.363 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.077) 0:04:46.590 ************ 2020-05-24T13:45:55.411 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.444 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.080) 0:04:46.671 ************ 2020-05-24T13:45:55.466 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-05-24T13:45:55.468 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice ntpdate.service -.mount tmp.mount sntp.service syslog.target basic.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2020-05-24T13:45:55.470 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2020-05-24T13:45:55.471 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:45:55.508 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.063) 0:04:46.735 ************ 2020-05-24T13:45:55.518 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-05-24T13:45:55.520 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.559 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.050) 0:04:46.786 ************ 2020-05-24T13:45:55.568 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:55.570 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-05-24T13:45:55.603 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.043) 0:04:46.830 ************ 2020-05-24T13:45:55.612 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-05-24T13:45:55.613 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.649 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.046) 0:04:46.876 ************ 2020-05-24T13:45:55.657 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:55.659 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.697 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.044) 0:04:46.921 ************ 2020-05-24T13:45:55.706 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-05-24T13:45:55.707 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:55.746 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:55 +0000 (0:00:00.051) 0:04:46.973 ************ 2020-05-24T13:45:56.258 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-05-24T13:45:56.260 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2020-05-24T13:45:56.298 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:56 +0000 (0:00:00.552) 0:04:47.525 ************ 2020-05-24T13:45:56.330 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-05-24T13:45:56.332 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:56.369 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:56 +0000 (0:00:00.070) 0:04:47.596 ************ 2020-05-24T13:45:56.400 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-05-24T13:45:56.401 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:56.436 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:56 +0000 (0:00:00.066) 0:04:47.662 ************ 2020-05-24T13:45:56.612 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-05-24T13:45:56.614 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2020-05-24T13:45:56.648 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:56 +0000 (0:00:00.212) 0:04:47.875 ************ 2020-05-24T13:45:56.687 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-05-24T13:45:56.689 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:56.722 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:56 +0000 (0:00:00.074) 0:04:47.949 ************ 2020-05-24T13:45:56.762 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-05-24T13:45:56.764 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:56.798 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:56 +0000 (0:00:00.076) 0:04:48.025 ************ 2020-05-24T13:45:57.134 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-05-24T13:45:57.136 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-05-24T13:45:57.174 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:57 +0000 (0:00:00.374) 0:04:48.399 ************ 2020-05-24T13:45:57.451 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2020-05-24T13:45:57.487 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:57 +0000 (0:00:00.314) 0:04:48.714 ************ 2020-05-24T13:45:57.518 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-05-24T13:45:57.520 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:57.559 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:57 +0000 (0:00:00.066) 0:04:48.780 ************ 2020-05-24T13:45:58.082 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sntp.service system.slice basic.target syslog.target systemd-journald.socket ntpdate.service tmp.mount -.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target chronyd.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": " 2020-05-24T13:45:58.084 INFO:teuthology.task.ansible.out:0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551 2020-05-24T13:45:58.085 INFO:teuthology.task.ansible.out:615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:45:58.130 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:58 +0000 (0:00:00.576) 0:04:49.357 ************ 2020-05-24T13:45:58.139 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-05-24T13:45:58.140 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-05-24T13:45:58.185 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:58 +0000 (0:00:00.045) 0:04:49.403 ************ TASK [testnode : set_fact] ***************************************************** ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-05-24T13:45:58.223 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:58 +0000 (0:00:00.047) 0:04:49.450 ************ 2020-05-24T13:45:58.252 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:58.254 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:58.287 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:58 +0000 (0:00:00.064) 0:04:49.514 ************ 2020-05-24T13:45:59.261 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-05-24T13:45:59.263 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2020-05-24T13:45:59.297 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:59 +0000 (0:00:01.009) 0:04:50.524 ************ 2020-05-24T13:45:59.338 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:59.375 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:59 +0000 (0:00:00.076) 0:04:50.601 ************ 2020-05-24T13:45:59.403 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-05-24T13:45:59.405 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:59.453 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:59 +0000 (0:00:00.078) 0:04:50.680 ************ 2020-05-24T13:45:59.797 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-05-24T13:45:59.843 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:59 +0000 (0:00:00.389) 0:04:51.070 ************ 2020-05-24T13:45:59.974 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 20.1.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 20.1.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-05-24T13:46:00.003 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.159) 0:04:51.230 ************ 2020-05-24T13:46:00.040 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-05-24T13:46:00.042 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:00.076 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.073) 0:04:51.303 ************ 2020-05-24T13:46:00.131 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-05-24T13:46:00.133 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-05-24T13:46:00.187 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.110) 0:04:51.414 ************ 2020-05-24T13:46:00.517 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-05-24T13:46:00.519 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-05-24T13:46:00.549 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.362) 0:04:51.776 ************ 2020-05-24T13:46:00.581 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:00.616 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.066) 0:04:51.842 ************ 2020-05-24T13:46:00.657 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-05-24T13:46:00.658 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-05-24T13:46:00.694 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.078) 0:04:51.920 ************ 2020-05-24T13:46:00.732 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-05-24T13:46:00.734 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:00.771 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.076) 0:04:51.997 ************ 2020-05-24T13:46:00.792 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-05-24T13:46:00.794 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-05-24T13:46:00.829 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:00 +0000 (0:00:00.058) 0:04:52.056 ************ 2020-05-24T13:46:01.094 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-05-24T13:46:01.096 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-05-24T13:46:01.136 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:01 +0000 (0:00:00.307) 0:04:52.363 ************ 2020-05-24T13:46:01.457 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-05-24T13:46:01.460 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-05-24T13:46:01.494 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:01 +0000 (0:00:00.357) 0:04:52.721 ************ 2020-05-24T13:46:01.524 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-05-24T13:46:01.526 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.557 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:01 +0000 (0:00:00.063) 0:04:52.784 ************ 2020-05-24T13:46:01.595 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-05-24T13:46:01.597 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.608 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.622 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.653 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:01 +0000 (0:00:00.095) 0:04:52.880 ************ 2020-05-24T13:46:01.708 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-05-24T13:46:01.709 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.720 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.726 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi193.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.733 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.778 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:01 +0000 (0:00:00.124) 0:04:53.005 ************ 2020-05-24T13:46:01.822 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:46:01.824 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.858 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:01 +0000 (0:00:00.079) 0:04:53.085 ************ 2020-05-24T13:46:01.896 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-05-24T13:46:01.898 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:01.947 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:01 +0000 (0:00:00.088) 0:04:53.173 ************ 2020-05-24T13:46:01.986 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-05-24T13:46:01.989 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:02.034 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.087) 0:04:53.260 ************ 2020-05-24T13:46:02.076 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:02.113 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.078) 0:04:53.339 ************ 2020-05-24T13:46:02.144 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:02.190 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.077) 0:04:53.417 ************ 2020-05-24T13:46:02.225 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-05-24T13:46:02.227 INFO:teuthology.task.ansible.out:skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:02.278 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.087) 0:04:53.504 ************ 2020-05-24T13:46:02.320 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:02.358 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.080) 0:04:53.585 ************ 2020-05-24T13:46:02.393 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi193.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:02.444 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.085) 0:04:53.671 ************ 2020-05-24T13:46:02.454 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-05-24T13:46:02.455 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 20.1.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 20.1.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-05-24T13:46:02.495 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.051) 0:04:53.722 ************ 2020-05-24T13:46:02.524 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-05-24T13:46:02.526 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:02.561 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.063) 0:04:53.786 ************ 2020-05-24T13:46:02.597 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-05-24T13:46:02.599 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-05-24T13:46:02.640 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:02 +0000 (0:00:00.080) 0:04:53.867 ************ 2020-05-24T13:46:02.806 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi193.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.013390", "end": "2020-05-24 13:46:02.747721", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-05-24 13:46:02.734331", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:02.974 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi019.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-05-24T13:46:03.007 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.367) 0:04:54.234 ************ 2020-05-24T13:46:03.041 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:03.074 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.066) 0:04:54.301 ************ 2020-05-24T13:46:03.097 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-24T13:46:03.099 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006414", "end": "2020-05-24 13:46:03.059748", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-05-24 13:46:03.053334", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:03.114 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi019.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-05-24T13:46:03.149 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.074) 0:04:54.376 ************ 2020-05-24T13:46:03.192 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-05-24T13:46:03.194 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:03.234 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.085) 0:04:54.461 ************ 2020-05-24T13:46:03.370 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-24T13:46:03.372 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005439", "end": "2020-05-24 13:46:03.331639", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:46:03.326200", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:03.562 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-05-24T13:46:03.563 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-05-24T13:46:03.597 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.362) 0:04:54.823 ************ 2020-05-24T13:46:03.667 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-24T13:46:03.669 INFO:teuthology.task.ansible.out:ok: [smithi193.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007718", "end": "2020-05-24 13:46:03.625135", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-05-24 13:46:03.617417", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:03.721 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.124) 0:04:54.948 ************ 2020-05-24T13:46:03.764 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-05-24T13:46:03.800 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.079) 0:04:55.027 ************ 2020-05-24T13:46:03.898 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-05-24T13:46:03.900 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-05-24T13:46:03.935 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:03 +0000 (0:00:00.135) 0:04:55.162 ************ 2020-05-24T13:46:03.967 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-05-24T13:46:03.968 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.005 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.069) 0:04:55.232 ************ 2020-05-24T13:46:04.045 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-05-24T13:46:04.048 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.058 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.069 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.071 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.083 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.120 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.114) 0:04:55.347 ************ 2020-05-24T13:46:04.133 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-05-24T13:46:04.139 INFO:teuthology.task.ansible.out:changed: [smithi193.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:46:04.175 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-05-24T13:46:04.176 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.178 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.211 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.213 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.215 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.246 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.125) 0:04:55.473 ************ 2020-05-24T13:46:04.288 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:46:04.289 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.331 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.084) 0:04:55.557 ************ 2020-05-24T13:46:04.370 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-05-24T13:46:04.372 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.405 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.074) 0:04:55.632 ************ 2020-05-24T13:46:04.446 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-05-24T13:46:04.448 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.484 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.078) 0:04:55.711 ************ 2020-05-24T13:46:04.523 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-05-24T13:46:04.524 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.561 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.077) 0:04:55.788 ************ 2020-05-24T13:46:04.600 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-05-24T13:46:04.602 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.643 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.081) 0:04:55.870 ************ 2020-05-24T13:46:04.674 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-05-24T13:46:04.676 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.712 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.069) 0:04:55.939 ************ 2020-05-24T13:46:04.743 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-05-24T13:46:04.748 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.780 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.068) 0:04:56.007 ************ 2020-05-24T13:46:04.809 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-05-24T13:46:04.811 INFO:teuthology.task.ansible.out:skipping: [smithi019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:46:04.849 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:04 +0000 (0:00:00.067) 0:04:56.074 ************ 2020-05-24T13:46:05.209 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-24T13:46:05.211 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.016631", "end": "2020-05-24 13:46:05.174472", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-05-24 13:46:05.157841", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:05.533 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007743", "end": "2020-05-24 13:46:05.498236", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-05-24 13:46:05.490493", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:05.866 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007706", "end": "2020-05-24 13:46:05.811458", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:46:05.803752", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:06.174 INFO:teuthology.task.ansible.out:ok: [smithi019.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008607", "end": "2020-05-24 13:46:06.135171", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-05-24 13:46:06.126564", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:46:06.209 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:06 +0000 (0:00:01.361) 0:04:57.436 ************ 2020-05-24T13:46:06.250 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-05-24T13:46:06.283 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:46:06 +0000 (0:00:00.073) 0:04:57.509 ************ 2020-05-24T13:46:06.613 INFO:teuthology.task.ansible.out: 2020-05-24T13:46:06.614 INFO:teuthology.task.ansible.out:TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-05-24T13:46:06.615 INFO:teuthology.task.ansible.out:changed: [smithi019.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:46:06.730 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-05-24T13:46:06.736 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi019.front.sepia.ceph.com : ok=120 changed=42 unreachable=0 failed=0 skipped=164 rescued=0 ignored=3 smithi193.front.sepia.ceph.com : ok=120 changed=42 unreachable=0 failed=0 skipped=164 rescued=0 ignored=3 2020-05-24T13:46:06.737 INFO:teuthology.task.ansible.out: Sunday 24 May 2020 13:46:06 +0000 (0:00:00.453) 0:04:57.963 ************ =============================================================================== users : Create all admin users with sudo access. ----------------------- 41.44s users : Update authorized_keys using the keys repo --------------------- 37.90s common : nrpe - Ensure SELinux policy is up to date -------------------- 22.33s testnode : Install packages -------------------------------------------- 14.07s users : Remove revoked users ------------------------------------------- 12.40s common : Make sure smartmontools is installed -------------------------- 12.00s common : nrpe - Remove SELinux policy package --------------------------- 6.34s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 6.01s testnode : Zap all non-root disks --------------------------------------- 5.85s common : nrpe - Load SELinux policy package ----------------------------- 5.71s testnode : Ensure ceph packages are not present. ------------------------ 5.36s Gathering Facts --------------------------------------------------------- 2.49s common : Configure epel repos in /etc/yum.repos.d/ ---------------------- 1.90s 2020-05-24T13:46:06.737 INFO:teuthology.task.ansible.out:testnode : Rebuild rpmdb ------------------------------------------------ 1.84s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.71s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.45s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.36s com 2020-05-24T13:46:06.739 INFO:teuthology.task.ansible.out:mon : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.12s testnode : Configure /etc/sudoers. -------------------------------------- 1.08s common : nrpe - Copy SELinux type enforcement file ---------------------- 1.06s 2020-05-24T13:46:07.172 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2020-05-24T13:46:07.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:46:07.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:46:07.290 INFO:teuthology.orchestra.run.smithi019:> true 2020-05-24T13:46:07.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:46:07.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:46:07.804 INFO:teuthology.orchestra.run.smithi193:> true 2020-05-24T13:46:08.197 INFO:teuthology.run_tasks:Running task clock... 2020-05-24T13:46:08.234 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-05-24T13:46:08.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-05-24T13:46:08.236 INFO:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-05-24T13:46:08.382 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-05-24T13:46:15.471 INFO:teuthology.orchestra.run.smithi019.stdout:ntpd: time slew -0.001229s 2020-05-24T13:46:15.510 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit not found. 2020-05-24T13:46:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: remote refid st t when poll reach delay offset jitter 2020-05-24T13:46:15.600 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================== 2020-05-24T13:46:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:15.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-05-24T13:46:15.601 INFO:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-05-24T13:46:15.740 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-05-24T13:46:22.814 INFO:teuthology.orchestra.run.smithi193.stdout:ntpd: time slew -0.001381s 2020-05-24T13:46:22.850 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit not found. 2020-05-24T13:46:22.929 INFO:teuthology.orchestra.run.smithi193.stdout: remote refid st t when poll reach delay offset jitter 2020-05-24T13:46:22.930 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================== 2020-05-24T13:46:22.930 INFO:teuthology.orchestra.run.smithi193.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:22.930 INFO:teuthology.orchestra.run.smithi193.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:22.931 INFO:teuthology.orchestra.run.smithi193.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:22.931 INFO:teuthology.orchestra.run.smithi193.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:46:22.932 INFO:teuthology.run_tasks:Running task install... 2020-05-24T13:46:22.947 DEBUG:teuthology.task.install:project ceph 2020-05-24T13:46:22.948 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'}} 2020-05-24T13:46:22.949 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'} 2020-05-24T13:46:22.949 INFO:teuthology.task.install:Using flavor: notcmalloc 2020-05-24T13:46:22.976 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2020-05-24T13:46:22.977 INFO:teuthology.task.install:extra packages: [] 2020-05-24T13:46:22.977 INFO:teuthology.orchestra.run.smithi019:> sudo lsb_release -is 2020-05-24T13:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS 2020-05-24T13:46:23.146 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:46:23.146 INFO:teuthology.orchestra.run.smithi193:> sudo lsb_release -is 2020-05-24T13:46:23.147 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'debuginfo': None, 'flavor': 'notcmalloc', 'downgrade_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-05-24T13:46:23.148 INFO:teuthology.orchestra.run.smithi019:> sudo lsb_release -is 2020-05-24T13:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS 2020-05-24T13:46:23.289 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:46:23.290 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:46:23.304 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS 2020-05-24T13:46:23.306 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:46:23.307 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'debuginfo': None, 'flavor': 'notcmalloc', 'downgrade_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-05-24T13:46:23.307 INFO:teuthology.orchestra.run.smithi193:> sudo lsb_release -is 2020-05-24T13:46:23.424 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS 2020-05-24T13:46:23.424 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:46:23.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:46:23.548 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/ 2020-05-24T13:46:23.549 INFO:teuthology.task.install.rpm:Package version is 14.2.9-537.g6fe217b2dd 2020-05-24T13:46:23.683 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/ 2020-05-24T13:46:23.683 INFO:teuthology.task.install.rpm:Package version is 14.2.9-537.g6fe217b2dd 2020-05-24T13:46:23.748 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-05-24T13:46:23.749 INFO:teuthology.orchestra.run.smithi019:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-05-24T13:46:23.871 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2020-05-24T13:46:23.871 INFO:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6fe217b2dddadf1baae6a015a57f83f4283a6078/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-05-24T13:46:23.892 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-05-24T13:46:23.893 INFO:teuthology.orchestra.run.smithi193:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-05-24T13:46:23.987 INFO:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-05-24T13:46:24.014 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2020-05-24T13:46:24.014 INFO:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6fe217b2dddadf1baae6a015a57f83f4283a6078/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-05-24T13:46:24.113 INFO:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-05-24T13:46:24.118 INFO:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-05-24T13:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2020-05-24T13:46:24.206 INFO:teuthology.orchestra.run.smithi019:> sudo yum clean all 2020-05-24T13:46:24.249 INFO:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-05-24T13:46:24.344 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2020-05-24T13:46:24.369 INFO:teuthology.orchestra.run.smithi193:> sudo yum clean all 2020-05-24T13:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2020-05-24T13:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: : lab-extras updates 2020-05-24T13:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout:Cleaning up list of fastest mirrors 2020-05-24T13:46:24.467 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw 2020-05-24T13:46:24.623 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:24.665 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:24.670 INFO:teuthology.orchestra.run.smithi193.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2020-05-24T13:46:24.670 INFO:teuthology.orchestra.run.smithi193.stdout: : lab-extras updates 2020-05-24T13:46:24.688 INFO:teuthology.orchestra.run.smithi193.stdout:Cleaning up list of fastest mirrors 2020-05-24T13:46:24.701 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw 2020-05-24T13:46:24.705 INFO:teuthology.orchestra.run.smithi019.stdout:Determining fastest mirrors 2020-05-24T13:46:24.934 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:24.969 INFO:teuthology.orchestra.run.smithi193.stdout:Determining fastest mirrors 2020-05-24T13:46:25.251 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:25.251 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:46:25.252 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:25.253 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:25.618 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:25.619 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:46:25.619 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:25.620 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:31.668 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:32.031 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:46:32.032 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:46:32.032 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.046 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-base = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.156 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-selinux = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.157 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: librgw2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.158 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: librados2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.160 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.160 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.162 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.163 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libceph-common.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.163 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:46:32.163 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.172 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-common = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.174 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: librbd1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.175 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libcephfs2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.176 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-requests for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.178 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.179 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.180 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.183 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2020-05-24T13:46:32.183 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2020-05-24T13:46:32.184 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:32.185 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:32.187 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.188 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:46:32.188 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.192 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-rgw = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.193 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libradosstriper1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.194 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-cephfs = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.195 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-ceph-argparse = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.196 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-rados = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.197 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-rbd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.198 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.199 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.200 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.201 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.202 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:32.202 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2020-05-24T13:46:32.202 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.203 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:32.203 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:32.204 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2020-05-24T13:46:32.205 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:32.206 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:32.206 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-requests.noarch 0:2.6.0-9.el7_8 will be installed 2020-05-24T13:46:32.207 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-9.el7_8.noarch 2020-05-24T13:46:32.207 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:46:32.208 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2020-05-24T13:46:32.209 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.210 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2020-05-24T13:46:32.211 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.211 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.212 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2020-05-24T13:46:32.213 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.213 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.213 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:32.214 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2020-05-24T13:46:32.214 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2020-05-24T13:46:33.078 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:33.081 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:33.161 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:46:33.162 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:46:33.165 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:46:33.165 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:46:33.165 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository 2020-05-24T13:46:33.166 INFO:teuthology.orchestra.run.smithi193.stdout: Size 2020-05-24T13:46:33.166 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:46:33.166 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:46:33.166 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.8 M 2020-05-24T13:46:33.166 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:46:33.166 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.4 M 2020-05-24T13:46:33.167 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 19 M 2020-05-24T13:46:33.167 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 20 k 2020-05-24T13:46:33.167 INFO:teuthology.orchestra.run.smithi193.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2020-05-24T13:46:33.167 INFO:teuthology.orchestra.run.smithi193.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2020-05-24T13:46:33.168 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 501 k 2020-05-24T13:46:33.168 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2020-05-24T13:46:33.168 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2020-05-24T13:46:33.168 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 363 k 2020-05-24T13:46:33.169 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2020-05-24T13:46:33.169 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.1 M 2020-05-24T13:46:33.169 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2020-05-24T13:46:33.169 INFO:teuthology.orchestra.run.smithi193.stdout: python-ceph-argparse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 35 k 2020-05-24T13:46:33.169 INFO:teuthology.orchestra.run.smithi193.stdout: python-cephfs x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 111 k 2020-05-24T13:46:33.169 INFO:teuthology.orchestra.run.smithi193.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2020-05-24T13:46:33.170 INFO:teuthology.orchestra.run.smithi193.stdout: python-rados x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 198 k 2020-05-24T13:46:33.170 INFO:teuthology.orchestra.run.smithi193.stdout: python-rbd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 192 k 2020-05-24T13:46:33.170 INFO:teuthology.orchestra.run.smithi193.stdout: python-requests noarch 2.6.0-9.el7_8 updates 94 k 2020-05-24T13:46:33.170 INFO:teuthology.orchestra.run.smithi193.stdout: python-rgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 77 k 2020-05-24T13:46:33.170 INFO:teuthology.orchestra.run.smithi193.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2020-05-24T13:46:33.171 INFO:teuthology.orchestra.run.smithi193.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2020-05-24T13:46:33.171 INFO:teuthology.orchestra.run.smithi193.stdout:Updating for dependencies: 2020-05-24T13:46:33.171 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 3.5 M 2020-05-24T13:46:33.171 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:46:33.171 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:46:33.171 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:46:33.172 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:46:33.172 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+21 Dependent packages) 2020-05-24T13:46:33.172 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade ( 2 Dependent packages) 2020-05-24T13:46:33.172 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:46:33.172 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 43 M 2020-05-24T13:46:33.173 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:46:33.173 INFO:teuthology.orchestra.run.smithi193.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:46:33.456 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:46:33.457 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:46:33.457 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.473 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-base = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.591 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-selinux = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.599 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: librgw2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.599 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: librados2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.599 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.600 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.600 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.600 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libceph-common.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.601 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:46:33.601 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.609 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-common = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.611 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: librbd1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.612 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libcephfs2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.613 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-requests for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.615 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.616 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.617 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.620 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2020-05-24T13:46:33.621 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2020-05-24T13:46:33.621 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:33.622 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:33.625 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.626 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:46:33.626 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.630 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-rgw = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.632 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libradosstriper1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-cephfs = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-ceph-argparse = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-rados = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-rbd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.638 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.639 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.640 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.641 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:33.642 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2020-05-24T13:46:33.642 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.643 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:33.643 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:33.644 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2020-05-24T13:46:33.644 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:33.645 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:33.645 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-requests.noarch 0:2.6.0-9.el7_8 will be installed 2020-05-24T13:46:33.646 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-9.el7_8.noarch 2020-05-24T13:46:33.646 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:46:33.647 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2020-05-24T13:46:33.648 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.648 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2020-05-24T13:46:33.649 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.650 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.651 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2020-05-24T13:46:33.651 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.652 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.653 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:33.653 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2020-05-24T13:46:33.653 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2020-05-24T13:46:34.567 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:34.654 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:46:34.654 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:46:34.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:46:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:46:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository 2020-05-24T13:46:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: Size 2020-05-24T13:46:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:46:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:46:34.660 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.8 M 2020-05-24T13:46:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:46:34.660 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.4 M 2020-05-24T13:46:34.660 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 19 M 2020-05-24T13:46:34.660 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 20 k 2020-05-24T13:46:34.661 INFO:teuthology.orchestra.run.smithi019.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2020-05-24T13:46:34.661 INFO:teuthology.orchestra.run.smithi019.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2020-05-24T13:46:34.661 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 501 k 2020-05-24T13:46:34.661 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2020-05-24T13:46:34.661 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2020-05-24T13:46:34.662 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 363 k 2020-05-24T13:46:34.662 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2020-05-24T13:46:34.662 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.1 M 2020-05-24T13:46:34.662 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2020-05-24T13:46:34.662 INFO:teuthology.orchestra.run.smithi019.stdout: python-ceph-argparse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 35 k 2020-05-24T13:46:34.662 INFO:teuthology.orchestra.run.smithi019.stdout: python-cephfs x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 111 k 2020-05-24T13:46:34.664 INFO:teuthology.orchestra.run.smithi019.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2020-05-24T13:46:34.664 INFO:teuthology.orchestra.run.smithi019.stdout: python-rados x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 198 k 2020-05-24T13:46:34.664 INFO:teuthology.orchestra.run.smithi019.stdout: python-rbd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 192 k 2020-05-24T13:46:34.664 INFO:teuthology.orchestra.run.smithi019.stdout: python-requests noarch 2.6.0-9.el7_8 updates 94 k 2020-05-24T13:46:34.664 INFO:teuthology.orchestra.run.smithi019.stdout: python-rgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 77 k 2020-05-24T13:46:34.665 INFO:teuthology.orchestra.run.smithi019.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2020-05-24T13:46:34.665 INFO:teuthology.orchestra.run.smithi019.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2020-05-24T13:46:34.665 INFO:teuthology.orchestra.run.smithi019.stdout:Updating for dependencies: 2020-05-24T13:46:34.665 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 3.5 M 2020-05-24T13:46:34.665 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:46:34.666 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:46:34.666 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:46:34.666 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:46:34.666 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+21 Dependent packages) 2020-05-24T13:46:34.666 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade ( 2 Dependent packages) 2020-05-24T13:46:34.667 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:46:34.667 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 43 M 2020-05-24T13:46:34.667 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:46:34.668 INFO:teuthology.orchestra.run.smithi019.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:46:35.317 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:35.318 INFO:teuthology.orchestra.run.smithi193.stdout:Total 20 MB/s | 43 MB 00:02 2020-05-24T13:46:35.329 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:46:35.394 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:46:35.576 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:46:35.576 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:46:36.212 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:46:36.472 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/26 2020-05-24T13:46:36.720 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:46:37.057 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:46:37.481 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 5/26 2020-05-24T13:46:37.534 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:37.535 INFO:teuthology.orchestra.run.smithi019.stdout:Total 15 MB/s | 43 MB 00:02 2020-05-24T13:46:37.555 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:46:37.641 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:46:38.202 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:46:38.203 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:46:38.243 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 6/26 2020-05-24T13:46:38.574 INFO:teuthology.orchestra.run.smithi193.stdout: Updating : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:46:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:46:39.403 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:46:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/26 2020-05-24T13:46:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:46:39.813 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 9/26 2020-05-24T13:46:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:46:40.354 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 10/26 2020-05-24T13:46:40.486 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 5/26 2020-05-24T13:46:40.695 INFO:teuthology.orchestra.run.smithi193.stdout: Updating : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:46:40.993 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 12/26 2020-05-24T13:46:41.251 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 13/26 2020-05-24T13:46:41.304 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 6/26 2020-05-24T13:46:41.589 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:46:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: Updating : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:46:41.884 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 15/26 2020-05-24T13:46:42.188 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 16/26 2020-05-24T13:46:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:46:42.510 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 17/26 2020-05-24T13:46:42.822 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 9/26 2020-05-24T13:46:42.869 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/26 2020-05-24T13:46:43.248 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 19/26 2020-05-24T13:46:43.635 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 10/26 2020-05-24T13:46:43.953 INFO:teuthology.orchestra.run.smithi019.stdout: Updating : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:46:44.230 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 12/26 2020-05-24T13:46:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 13/26 2020-05-24T13:46:44.856 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:46:45.165 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 15/26 2020-05-24T13:46:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 16/26 2020-05-24T13:46:45.613 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-requests-2.6.0-9.el7_8.noarch 20/26 2020-05-24T13:46:45.857 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 17/26 2020-05-24T13:46:46.217 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/26 2020-05-24T13:46:46.564 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 19/26 2020-05-24T13:46:46.735 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:46:47.247 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 22/26 2020-05-24T13:46:49.005 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-requests-2.6.0-9.el7_8.noarch 20/26 2020-05-24T13:46:50.163 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:46:50.699 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 22/26 2020-05-24T13:46:56.438 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:46:56.827 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 24/26 2020-05-24T13:46:57.085 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:46:57.387 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:46:57.507 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:46:57.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 2/26 2020-05-24T13:46:57.750 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:46:57.851 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:46:57.934 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 5/26 2020-05-24T13:46:58.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 6/26 2020-05-24T13:46:58.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:46:58.202 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:46:58.312 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 9/26 2020-05-24T13:46:58.422 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 10/26 2020-05-24T13:46:58.521 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:46:58.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 12/26 2020-05-24T13:46:58.722 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 13/26 2020-05-24T13:46:58.831 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:46:58.948 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 15/26 2020-05-24T13:46:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 16/26 2020-05-24T13:46:59.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 17/26 2020-05-24T13:46:59.938 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 18/26 2020-05-24T13:47:00.034 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 19/26 2020-05-24T13:47:00.126 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 20/26 2020-05-24T13:47:00.219 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:47:00.302 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-requests-2.6.0-9.el7_8.noarch 22/26 2020-05-24T13:47:00.420 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:47:00.545 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 24/26 2020-05-24T13:47:00.547 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:47:00.668 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:47:00.937 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:47:00.937 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:00.937 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:47:00.938 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.938 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:00.938 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:47:00.938 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.938 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.939 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.939 INFO:teuthology.orchestra.run.smithi193.stdout: leveldb.x86_64 0:1.12.0-11.el7 2020-05-24T13:47:00.939 INFO:teuthology.orchestra.run.smithi193.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2020-05-24T13:47:00.939 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.939 INFO:teuthology.orchestra.run.smithi193.stdout: liboath.x86_64 0:2.6.2-1.el7 2020-05-24T13:47:00.940 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2020-05-24T13:47:00.940 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.940 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2020-05-24T13:47:00.940 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.940 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2020-05-24T13:47:00.941 INFO:teuthology.orchestra.run.smithi193.stdout: python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.941 INFO:teuthology.orchestra.run.smithi193.stdout: python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.941 INFO:teuthology.orchestra.run.smithi193.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2020-05-24T13:47:00.941 INFO:teuthology.orchestra.run.smithi193.stdout: python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.941 INFO:teuthology.orchestra.run.smithi193.stdout: python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.942 INFO:teuthology.orchestra.run.smithi193.stdout: python-requests.noarch 0:2.6.0-9.el7_8 2020-05-24T13:47:00.943 INFO:teuthology.orchestra.run.smithi193.stdout: python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.943 INFO:teuthology.orchestra.run.smithi193.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2020-05-24T13:47:00.943 INFO:teuthology.orchestra.run.smithi193.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2020-05-24T13:47:00.944 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:00.944 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Updated: 2020-05-24T13:47:00.944 INFO:teuthology.orchestra.run.smithi193.stdout: librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.944 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:00.944 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:00.944 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:47:01.053 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 24/26 2020-05-24T13:47:01.066 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-test 2020-05-24T13:47:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:47:01.332 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:01.369 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:01.370 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:01.370 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:01.371 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:01.371 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:47:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:47:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 2/26 2020-05-24T13:47:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:47:02.009 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:47:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 5/26 2020-05-24T13:47:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 6/26 2020-05-24T13:47:02.333 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:47:02.334 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:02.334 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:02.353 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:02.385 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: jq for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:02.386 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: socat for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:02.387 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:02.387 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package jq.x86_64 0:1.6-1.el7 will be installed 2020-05-24T13:47:02.388 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.6-1.el7.x86_64 2020-05-24T13:47:02.388 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2020-05-24T13:47:02.390 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2020-05-24T13:47:02.393 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:02.393 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2020-05-24T13:47:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:47:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:47:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 9/26 2020-05-24T13:47:02.754 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 10/26 2020-05-24T13:47:02.756 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:02.855 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:47:02.955 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 12/26 2020-05-24T13:47:02.978 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:02.979 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:47:02.979 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:02.980 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:02.980 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository 2020-05-24T13:47:02.980 INFO:teuthology.orchestra.run.smithi193.stdout: Size 2020-05-24T13:47:02.980 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:02.980 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:47:02.981 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 34 M 2020-05-24T13:47:02.981 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:47:02.981 INFO:teuthology.orchestra.run.smithi193.stdout: jq x86_64 1.6-1.el7 epel 167 k 2020-05-24T13:47:02.981 INFO:teuthology.orchestra.run.smithi193.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2020-05-24T13:47:02.981 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2020-05-24T13:47:02.982 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2020-05-24T13:47:02.982 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:02.982 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:47:02.982 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:02.982 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:47:02.983 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:02.983 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 35 M 2020-05-24T13:47:02.983 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 121 M 2020-05-24T13:47:02.983 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:47:03.055 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 13/26 2020-05-24T13:47:03.164 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:47:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 15/26 2020-05-24T13:47:03.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 16/26 2020-05-24T13:47:03.516 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 17/26 2020-05-24T13:47:03.624 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 18/26 2020-05-24T13:47:03.733 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 19/26 2020-05-24T13:47:03.834 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 20/26 2020-05-24T13:47:03.934 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:47:04.035 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-requests-2.6.0-9.el7_8.noarch 22/26 2020-05-24T13:47:04.169 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:47:04.275 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 24/26 2020-05-24T13:47:04.275 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:47:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:47:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:04.647 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:47:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:04.648 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:47:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: leveldb.x86_64 0:1.12.0-11.el7 2020-05-24T13:47:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2020-05-24T13:47:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: liboath.x86_64 0:2.6.2-1.el7 2020-05-24T13:47:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2020-05-24T13:47:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2020-05-24T13:47:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2020-05-24T13:47:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2020-05-24T13:47:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: python-requests.noarch 0:2.6.0-9.el7_8 2020-05-24T13:47:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2020-05-24T13:47:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2020-05-24T13:47:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:04.658 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Updated: 2020-05-24T13:47:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:04.663 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:47:04.781 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-test 2020-05-24T13:47:05.083 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:05.128 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:05.719 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:05.719 INFO:teuthology.orchestra.run.smithi193.stdout:Total 13 MB/s | 35 MB 00:02 2020-05-24T13:47:05.722 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:47:05.749 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:47:05.785 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:05.807 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:47:05.808 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:47:06.136 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:47:06.136 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:06.137 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:06.158 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:06.190 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: jq for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:06.191 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: socat for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:06.192 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:06.192 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package jq.x86_64 0:1.6-1.el7 will be installed 2020-05-24T13:47:06.193 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.6-1.el7.x86_64 2020-05-24T13:47:06.195 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2020-05-24T13:47:06.196 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2020-05-24T13:47:06.199 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:06.199 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2020-05-24T13:47:06.531 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2020-05-24T13:47:06.584 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:47:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository 2020-05-24T13:47:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: Size 2020-05-24T13:47:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:47:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 34 M 2020-05-24T13:47:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:47:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: jq x86_64 1.6-1.el7 epel 167 k 2020-05-24T13:47:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2020-05-24T13:47:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2020-05-24T13:47:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2020-05-24T13:47:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:47:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:47:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 35 M 2020-05-24T13:47:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 121 M 2020-05-24T13:47:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:47:06.823 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2020-05-24T13:47:07.101 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2020-05-24T13:47:09.549 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:09.549 INFO:teuthology.orchestra.run.smithi019.stdout:Total 13 MB/s | 35 MB 00:02 2020-05-24T13:47:09.553 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:47:09.580 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:47:09.632 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:47:09.633 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:47:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2020-05-24T13:47:10.650 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2020-05-24T13:47:10.725 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : jq-1.6-1.el7.x86_64 4/5 2020-05-24T13:47:10.952 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2020-05-24T13:47:11.056 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 5/5 2020-05-24T13:47:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2020-05-24T13:47:11.279 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : jq-1.6-1.el7.x86_64 2/5 2020-05-24T13:47:11.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2020-05-24T13:47:11.488 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 4/5 2020-05-24T13:47:11.982 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2020-05-24T13:47:11.983 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:11.983 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:47:11.983 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:11.983 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:11.984 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:47:11.984 INFO:teuthology.orchestra.run.smithi193.stdout: jq.x86_64 0:1.6-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2020-05-24T13:47:11.984 INFO:teuthology.orchestra.run.smithi193.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2020-05-24T13:47:11.984 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:11.984 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:47:12.117 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph 2020-05-24T13:47:12.348 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:12.391 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:12.391 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:12.392 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:12.392 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:12.393 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:13.037 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:13.366 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:47:13.367 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:13.367 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:13.370 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-mgr = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.400 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-mds = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.401 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-mon = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.402 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-osd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.404 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:13.404 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:13.416 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:13.418 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.419 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.420 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.421 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.422 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:13.422 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:13.425 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:13.430 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:13.430 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2020-05-24T13:47:13.431 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2020-05-24T13:47:13.432 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2020-05-24T13:47:13.433 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:13.435 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:13.435 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:13.436 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:13.437 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:13.437 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2020-05-24T13:47:13.438 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2020-05-24T13:47:13.440 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2020-05-24T13:47:13.440 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:13.441 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2020-05-24T13:47:13.441 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:47:13.443 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:47:13.443 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2020-05-24T13:47:13.443 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2020-05-24T13:47:13.443 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2020-05-24T13:47:13.443 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2020-05-24T13:47:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2020-05-24T13:47:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2020-05-24T13:47:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2020-05-24T13:47:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2020-05-24T13:47:13.445 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:13.445 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2020-05-24T13:47:13.446 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2020-05-24T13:47:13.446 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:13.446 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2020-05-24T13:47:13.805 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:14.030 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:14.031 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:47:14.032 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:14.032 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:14.033 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository 2020-05-24T13:47:14.033 INFO:teuthology.orchestra.run.smithi193.stdout: Size 2020-05-24T13:47:14.034 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:14.034 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:47:14.034 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.0 k 2020-05-24T13:47:14.035 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:47:14.035 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.9 M 2020-05-24T13:47:14.035 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:47:14.036 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 4.4 M 2020-05-24T13:47:14.036 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 16 M 2020-05-24T13:47:14.036 INFO:teuthology.orchestra.run.smithi193.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2020-05-24T13:47:14.037 INFO:teuthology.orchestra.run.smithi193.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2020-05-24T13:47:14.037 INFO:teuthology.orchestra.run.smithi193.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2020-05-24T13:47:14.042 INFO:teuthology.orchestra.run.smithi193.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2020-05-24T13:47:14.043 INFO:teuthology.orchestra.run.smithi193.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2020-05-24T13:47:14.043 INFO:teuthology.orchestra.run.smithi193.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2020-05-24T13:47:14.044 INFO:teuthology.orchestra.run.smithi193.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2020-05-24T13:47:14.044 INFO:teuthology.orchestra.run.smithi193.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2020-05-24T13:47:14.044 INFO:teuthology.orchestra.run.smithi193.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2020-05-24T13:47:14.045 INFO:teuthology.orchestra.run.smithi193.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2020-05-24T13:47:14.045 INFO:teuthology.orchestra.run.smithi193.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2020-05-24T13:47:14.045 INFO:teuthology.orchestra.run.smithi193.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2020-05-24T13:47:14.046 INFO:teuthology.orchestra.run.smithi193.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2020-05-24T13:47:14.046 INFO:teuthology.orchestra.run.smithi193.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2020-05-24T13:47:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2020-05-24T13:47:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:14.047 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:47:14.048 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:14.048 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+19 Dependent packages) 2020-05-24T13:47:14.048 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:14.049 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 27 M 2020-05-24T13:47:14.049 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 86 M 2020-05-24T13:47:14.049 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:47:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : jq-1.6-1.el7.x86_64 4/5 2020-05-24T13:47:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 5/5 2020-05-24T13:47:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2020-05-24T13:47:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : jq-1.6-1.el7.x86_64 2/5 2020-05-24T13:47:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2020-05-24T13:47:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 4/5 2020-05-24T13:47:15.550 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:15.551 INFO:teuthology.orchestra.run.smithi193.stdout:Total 18 MB/s | 27 MB 00:01 2020-05-24T13:47:15.562 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:47:15.592 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:47:15.839 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:47:15.840 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:47:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2020-05-24T13:47:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:15.947 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:47:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:15.948 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:47:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: jq.x86_64 0:1.6-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2020-05-24T13:47:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2020-05-24T13:47:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:15.949 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:47:16.065 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph 2020-05-24T13:47:16.327 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:16.371 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:16.549 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 1/20 2020-05-24T13:47:16.891 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:47:17.053 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:17.194 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/20 2020-05-24T13:47:17.405 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:47:17.405 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:17.405 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:17.406 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-mgr = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.438 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-mds = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.439 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-mon = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.440 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-osd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.441 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:17.441 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:17.454 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:17.458 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.461 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:17.461 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:17.464 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:17.469 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:17.470 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2020-05-24T13:47:17.471 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2020-05-24T13:47:17.473 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2020-05-24T13:47:17.474 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:17.474 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:17.474 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:17.475 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:17.476 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:47:17.477 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2020-05-24T13:47:17.477 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2020-05-24T13:47:17.478 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2020-05-24T13:47:17.479 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:17.479 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2020-05-24T13:47:17.479 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:47:17.480 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:47:17.481 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2020-05-24T13:47:17.481 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2020-05-24T13:47:17.481 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2020-05-24T13:47:17.482 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2020-05-24T13:47:17.482 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2020-05-24T13:47:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2020-05-24T13:47:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2020-05-24T13:47:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2020-05-24T13:47:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2020-05-24T13:47:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2020-05-24T13:47:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2020-05-24T13:47:17.873 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:18.108 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:18.109 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:47:18.111 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:18.112 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:18.112 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository 2020-05-24T13:47:18.112 INFO:teuthology.orchestra.run.smithi019.stdout: Size 2020-05-24T13:47:18.113 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:18.113 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:47:18.113 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.0 k 2020-05-24T13:47:18.113 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:47:18.113 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.9 M 2020-05-24T13:47:18.114 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:47:18.114 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 4.4 M 2020-05-24T13:47:18.114 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 16 M 2020-05-24T13:47:18.114 INFO:teuthology.orchestra.run.smithi019.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2020-05-24T13:47:18.114 INFO:teuthology.orchestra.run.smithi019.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2020-05-24T13:47:18.115 INFO:teuthology.orchestra.run.smithi019.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2020-05-24T13:47:18.115 INFO:teuthology.orchestra.run.smithi019.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2020-05-24T13:47:18.115 INFO:teuthology.orchestra.run.smithi019.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2020-05-24T13:47:18.115 INFO:teuthology.orchestra.run.smithi019.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2020-05-24T13:47:18.115 INFO:teuthology.orchestra.run.smithi019.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2020-05-24T13:47:18.116 INFO:teuthology.orchestra.run.smithi019.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2020-05-24T13:47:18.116 INFO:teuthology.orchestra.run.smithi019.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2020-05-24T13:47:18.116 INFO:teuthology.orchestra.run.smithi019.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2020-05-24T13:47:18.116 INFO:teuthology.orchestra.run.smithi019.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2020-05-24T13:47:18.116 INFO:teuthology.orchestra.run.smithi019.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2020-05-24T13:47:18.116 INFO:teuthology.orchestra.run.smithi019.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2020-05-24T13:47:18.117 INFO:teuthology.orchestra.run.smithi019.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2020-05-24T13:47:18.117 INFO:teuthology.orchestra.run.smithi019.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2020-05-24T13:47:18.117 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:18.117 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:47:18.117 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:18.118 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+19 Dependent packages) 2020-05-24T13:47:18.118 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:18.119 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 27 M 2020-05-24T13:47:18.119 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 86 M 2020-05-24T13:47:18.119 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:47:19.027 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 4/20 2020-05-24T13:47:20.152 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:20.153 INFO:teuthology.orchestra.run.smithi019.stdout:Total 13 MB/s | 27 MB 00:02 2020-05-24T13:47:20.175 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:47:20.209 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 5/20 2020-05-24T13:47:20.221 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:47:20.486 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:47:20.486 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:47:20.590 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 6/20 2020-05-24T13:47:21.120 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 7/20 2020-05-24T13:47:21.294 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 1/20 2020-05-24T13:47:21.475 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 8/20 2020-05-24T13:47:21.623 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:47:21.905 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 9/20 2020-05-24T13:47:21.943 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/20 2020-05-24T13:47:22.390 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-mako-0.8.1-2.el7.noarch 10/20 2020-05-24T13:47:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 11/20 2020-05-24T13:47:23.546 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 12/20 2020-05-24T13:47:23.828 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-six-1.9.0-0.el7.noarch 13/20 2020-05-24T13:47:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 4/20 2020-05-24T13:47:24.247 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 14/20 2020-05-24T13:47:24.534 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 5/20 2020-05-24T13:47:24.537 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 15/20 2020-05-24T13:47:24.757 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 16/20 2020-05-24T13:47:24.922 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 6/20 2020-05-24T13:47:25.154 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 17/20 2020-05-24T13:47:25.423 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 7/20 2020-05-24T13:47:25.782 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 8/20 2020-05-24T13:47:25.847 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 18/20 2020-05-24T13:47:26.191 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 9/20 2020-05-24T13:47:26.247 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 19/20 2020-05-24T13:47:26.501 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:26.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 1/20 2020-05-24T13:47:26.673 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-mako-0.8.1-2.el7.noarch 10/20 2020-05-24T13:47:26.747 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:47:26.831 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 3/20 2020-05-24T13:47:26.923 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 4/20 2020-05-24T13:47:27.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 5/20 2020-05-24T13:47:27.132 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 6/20 2020-05-24T13:47:27.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 7/20 2020-05-24T13:47:27.350 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 8/20 2020-05-24T13:47:27.450 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 9/20 2020-05-24T13:47:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 11/20 2020-05-24T13:47:27.551 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 10/20 2020-05-24T13:47:27.660 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 11/20 2020-05-24T13:47:27.777 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 12/20 2020-05-24T13:47:27.809 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 12/20 2020-05-24T13:47:27.894 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 13/20 2020-05-24T13:47:28.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 14/20 2020-05-24T13:47:28.031 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-six-1.9.0-0.el7.noarch 13/20 2020-05-24T13:47:28.145 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 15/20 2020-05-24T13:47:28.233 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 16/20 2020-05-24T13:47:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 17/20 2020-05-24T13:47:28.413 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 18/20 2020-05-24T13:47:28.505 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 19/20 2020-05-24T13:47:28.506 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 14/20 2020-05-24T13:47:28.797 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 15/20 2020-05-24T13:47:28.942 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:28.943 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:28.943 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:47:28.943 INFO:teuthology.orchestra.run.smithi193.stdout: ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:28.943 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:28.944 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:47:28.944 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:28.944 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:28.944 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:28.944 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:28.945 INFO:teuthology.orchestra.run.smithi193.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2020-05-24T13:47:28.945 INFO:teuthology.orchestra.run.smithi193.stdout: python-beaker.noarch 0:1.5.4-10.el7 2020-05-24T13:47:28.945 INFO:teuthology.orchestra.run.smithi193.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2020-05-24T13:47:28.945 INFO:teuthology.orchestra.run.smithi193.stdout: python-mako.noarch 0:0.8.1-2.el7 2020-05-24T13:47:28.945 INFO:teuthology.orchestra.run.smithi193.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2020-05-24T13:47:28.946 INFO:teuthology.orchestra.run.smithi193.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2020-05-24T13:47:28.946 INFO:teuthology.orchestra.run.smithi193.stdout: python-pecan.noarch 0:0.4.5-2.el7 2020-05-24T13:47:28.946 INFO:teuthology.orchestra.run.smithi193.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2020-05-24T13:47:28.946 INFO:teuthology.orchestra.run.smithi193.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2020-05-24T13:47:28.946 INFO:teuthology.orchestra.run.smithi193.stdout: python-tempita.noarch 0:0.5.1-6.el7 2020-05-24T13:47:28.947 INFO:teuthology.orchestra.run.smithi193.stdout: python-webob.noarch 0:1.2.3-7.el7 2020-05-24T13:47:28.947 INFO:teuthology.orchestra.run.smithi193.stdout: python-webtest.noarch 0:1.3.4-6.el7 2020-05-24T13:47:28.948 INFO:teuthology.orchestra.run.smithi193.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2020-05-24T13:47:28.948 INFO:teuthology.orchestra.run.smithi193.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2020-05-24T13:47:28.948 INFO:teuthology.orchestra.run.smithi193.stdout: python2-six.noarch 0:1.9.0-0.el7 2020-05-24T13:47:28.949 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:28.949 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:47:29.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 16/20 2020-05-24T13:47:29.065 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr 2020-05-24T13:47:29.306 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:29.347 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:29.348 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:29.349 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:29.349 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:29.349 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:29.491 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 17/20 2020-05-24T13:47:29.986 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 18/20 2020-05-24T13:47:30.310 INFO:teuthology.orchestra.run.smithi193.stdout:Package 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:30.311 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do 2020-05-24T13:47:30.372 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-dashboard 2020-05-24T13:47:30.564 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:30.577 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 19/20 2020-05-24T13:47:30.598 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:30.599 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:30.600 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:30.600 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:30.600 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:30.800 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:30.901 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 1/20 2020-05-24T13:47:31.035 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:47:31.118 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 3/20 2020-05-24T13:47:31.199 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:31.229 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 4/20 2020-05-24T13:47:31.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 5/20 2020-05-24T13:47:31.438 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 6/20 2020-05-24T13:47:31.526 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:47:31.526 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:31.527 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:31.529 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:31.539 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 7/20 2020-05-24T13:47:31.560 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:31.561 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:31.562 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:31.562 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2020-05-24T13:47:31.562 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2020-05-24T13:47:31.564 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:31.564 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2020-05-24T13:47:31.648 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 8/20 2020-05-24T13:47:31.773 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 9/20 2020-05-24T13:47:31.874 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 10/20 2020-05-24T13:47:31.930 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:31.974 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 11/20 2020-05-24T13:47:32.075 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 12/20 2020-05-24T13:47:32.150 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:32.151 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:47:32.152 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:32.152 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:32.153 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2020-05-24T13:47:32.153 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:32.153 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:47:32.154 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 4.0 M 2020-05-24T13:47:32.154 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:47:32.154 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 18 k 2020-05-24T13:47:32.155 INFO:teuthology.orchestra.run.smithi193.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2020-05-24T13:47:32.155 INFO:teuthology.orchestra.run.smithi193.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2020-05-24T13:47:32.155 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:32.155 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:47:32.156 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:32.156 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:47:32.156 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:32.157 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 4.6 M 2020-05-24T13:47:32.157 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 20 M 2020-05-24T13:47:32.157 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:47:32.167 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 13/20 2020-05-24T13:47:32.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 14/20 2020-05-24T13:47:32.351 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 15/20 2020-05-24T13:47:32.476 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 16/20 2020-05-24T13:47:32.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 17/20 2020-05-24T13:47:32.694 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 18/20 2020-05-24T13:47:32.803 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 19/20 2020-05-24T13:47:32.940 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:32.940 INFO:teuthology.orchestra.run.smithi193.stdout:Total 5.9 MB/s | 4.6 MB 00:00 2020-05-24T13:47:32.945 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:47:32.953 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:47:33.121 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:47:33.122 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:47:33.357 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:33.358 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:33.358 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:47:33.358 INFO:teuthology.orchestra.run.smithi019.stdout: ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:33.358 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:33.358 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:47:33.359 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:33.359 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:33.359 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:33.359 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:33.359 INFO:teuthology.orchestra.run.smithi019.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2020-05-24T13:47:33.359 INFO:teuthology.orchestra.run.smithi019.stdout: python-beaker.noarch 0:1.5.4-10.el7 2020-05-24T13:47:33.360 INFO:teuthology.orchestra.run.smithi019.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2020-05-24T13:47:33.360 INFO:teuthology.orchestra.run.smithi019.stdout: python-mako.noarch 0:0.8.1-2.el7 2020-05-24T13:47:33.360 INFO:teuthology.orchestra.run.smithi019.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2020-05-24T13:47:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2020-05-24T13:47:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: python-pecan.noarch 0:0.4.5-2.el7 2020-05-24T13:47:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2020-05-24T13:47:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2020-05-24T13:47:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: python-tempita.noarch 0:0.5.1-6.el7 2020-05-24T13:47:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: python-webob.noarch 0:1.2.3-7.el7 2020-05-24T13:47:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: python-webtest.noarch 0:1.3.4-6.el7 2020-05-24T13:47:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2020-05-24T13:47:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2020-05-24T13:47:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python2-six.noarch 0:1.9.0-0.el7 2020-05-24T13:47:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:33.363 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:47:33.492 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr 2020-05-24T13:47:33.738 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 1/4 2020-05-24T13:47:33.753 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:33.794 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:33.795 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:33.796 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:33.796 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:33.796 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:34.157 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2020-05-24T13:47:34.459 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:34.804 INFO:teuthology.orchestra.run.smithi019.stdout:Package 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:34.804 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do 2020-05-24T13:47:34.865 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-dashboard 2020-05-24T13:47:35.063 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:35.103 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:35.104 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:35.105 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:35.105 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:35.105 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:35.358 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:35.698 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:35.753 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:35.814 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2020-05-24T13:47:35.922 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 2/4 2020-05-24T13:47:36.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:47:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:36.127 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:36.128 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:36.128 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:36.128 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2020-05-24T13:47:36.129 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2020-05-24T13:47:36.130 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:36.131 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2020-05-24T13:47:36.517 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:36.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:36.525 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:36.525 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:47:36.526 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:36.526 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:36.526 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:47:36.526 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:36.526 INFO:teuthology.orchestra.run.smithi193.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2020-05-24T13:47:36.527 INFO:teuthology.orchestra.run.smithi193.stdout: python-routes.noarch 0:1.13-2.el7 2020-05-24T13:47:36.527 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:36.527 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:47:36.656 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-diskprediction-cloud 2020-05-24T13:47:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:36.748 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:47:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:36.749 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2020-05-24T13:47:36.749 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:36.750 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:47:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 4.0 M 2020-05-24T13:47:36.750 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:47:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 18 k 2020-05-24T13:47:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2020-05-24T13:47:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2020-05-24T13:47:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:36.751 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:47:36.752 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:36.752 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:47:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:36.753 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 4.6 M 2020-05-24T13:47:36.754 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 20 M 2020-05-24T13:47:36.754 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:47:36.895 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:36.935 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:36.936 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:36.937 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:36.937 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:36.937 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:37.587 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:Total 4.6 MB/s | 4.6 MB 00:01 2020-05-24T13:47:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:47:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:47:37.909 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:47:37.910 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:37.910 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:37.955 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:47:37.956 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:47:38.281 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:38.463 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:38.463 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:47:38.467 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:38.467 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:38.467 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2020-05-24T13:47:38.468 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:38.468 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:47:38.468 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-cloud 2020-05-24T13:47:38.468 INFO:teuthology.orchestra.run.smithi193.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 83 k 2020-05-24T13:47:38.468 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:38.469 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:47:38.469 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:38.469 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2020-05-24T13:47:38.469 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:38.469 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 83 k 2020-05-24T13:47:38.470 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 412 k 2020-05-24T13:47:38.470 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:47:38.635 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 1/4 2020-05-24T13:47:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2020-05-24T13:47:39.385 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:47:39.389 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:47:39.444 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:47:39.445 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:47:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:40.598 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:40.675 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2020-05-24T13:47:40.767 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 2/4 2020-05-24T13:47:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:41.021 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:47:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:41.022 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:47:41.145 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-diskprediction-local 2020-05-24T13:47:41.360 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:41.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:41.361 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:47:41.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:41.361 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:41.361 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:47:41.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:41.362 INFO:teuthology.orchestra.run.smithi019.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2020-05-24T13:47:41.362 INFO:teuthology.orchestra.run.smithi019.stdout: python-routes.noarch 0:1.13-2.el7 2020-05-24T13:47:41.362 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:41.362 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:47:41.366 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:41.408 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:41.409 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:41.410 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:41.410 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:41.410 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:41.493 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-diskprediction-cloud 2020-05-24T13:47:41.691 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:41.729 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:41.742 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:41.743 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:41.744 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:41.744 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:42.050 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:42.353 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:42.380 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:47:42.381 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:42.382 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:42.384 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:42.415 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:42.415 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2020-05-24T13:47:42.422 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:42.423 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:42.425 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:42.426 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:42.426 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2020-05-24T13:47:42.426 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2020-05-24T13:47:42.428 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:42.429 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:42.430 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:42.430 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2020-05-24T13:47:42.690 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:47:42.691 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:42.691 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:42.795 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:43.021 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:43.022 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:47:43.023 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:43.023 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:43.023 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2020-05-24T13:47:43.023 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:43.024 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:47:43.024 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local 2020-05-24T13:47:43.024 INFO:teuthology.orchestra.run.smithi193.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 1.1 M 2020-05-24T13:47:43.024 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:47:43.024 INFO:teuthology.orchestra.run.smithi193.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2020-05-24T13:47:43.025 INFO:teuthology.orchestra.run.smithi193.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2020-05-24T13:47:43.025 INFO:teuthology.orchestra.run.smithi193.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2020-05-24T13:47:43.025 INFO:teuthology.orchestra.run.smithi193.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2020-05-24T13:47:43.025 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:43.025 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:47:43.026 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:43.026 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:47:43.026 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:43.026 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 12 M 2020-05-24T13:47:43.026 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 66 M 2020-05-24T13:47:43.027 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:47:43.082 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:43.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:43.275 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:47:43.276 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:43.276 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:43.276 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2020-05-24T13:47:43.277 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:43.277 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:47:43.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-cloud 2020-05-24T13:47:43.277 INFO:teuthology.orchestra.run.smithi019.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 83 k 2020-05-24T13:47:43.277 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:43.278 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:47:43.278 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:43.278 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2020-05-24T13:47:43.278 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:43.278 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 83 k 2020-05-24T13:47:43.279 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 412 k 2020-05-24T13:47:43.279 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:47:43.976 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:47:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:47:44.037 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:47:44.038 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:47:44.406 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:44.407 INFO:teuthology.orchestra.run.smithi193.stdout:Total 8.4 MB/s | 12 MB 00:01 2020-05-24T13:47:44.422 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:47:44.459 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:47:44.721 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:47:44.722 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:47:44.850 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:45.322 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:47:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:45.323 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:47:45.475 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-diskprediction-local 2020-05-24T13:47:45.726 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:45.772 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:45.935 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2020-05-24T13:47:46.408 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2020-05-24T13:47:46.442 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:46.794 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:47:46.794 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:46.795 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:46.797 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:46.829 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:46.830 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2020-05-24T13:47:46.839 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:46.839 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:46.841 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:46.841 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:46.841 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2020-05-24T13:47:46.842 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2020-05-24T13:47:46.843 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:46.845 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:46.845 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:46.845 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2020-05-24T13:47:47.232 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:47.464 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:47:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:47.464 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:47.464 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2020-05-24T13:47:47.465 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:47.465 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:47:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local 2020-05-24T13:47:47.465 INFO:teuthology.orchestra.run.smithi019.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 1.1 M 2020-05-24T13:47:47.465 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:47:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2020-05-24T13:47:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2020-05-24T13:47:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2020-05-24T13:47:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2020-05-24T13:47:47.466 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:47.467 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:47:47.467 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:47.467 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:47:47.469 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:47.470 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 12 M 2020-05-24T13:47:47.470 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 66 M 2020-05-24T13:47:47.471 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:47:48.129 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:48.884 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:48.885 INFO:teuthology.orchestra.run.smithi019.stdout:Total 8.2 MB/s | 12 MB 00:01 2020-05-24T13:47:48.901 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:47:48.933 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:47:48.997 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2020-05-24T13:47:49.091 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:47:49.092 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:47:49.318 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 5/5 2020-05-24T13:47:49.409 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2020-05-24T13:47:49.509 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 2/5 2020-05-24T13:47:49.593 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:49.919 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2020-05-24T13:47:50.094 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 4/5 2020-05-24T13:47:50.448 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2020-05-24T13:47:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2020-05-24T13:47:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:50.564 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:47:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:50.564 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:47:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2020-05-24T13:47:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2020-05-24T13:47:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:50.565 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:47:50.700 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-rook 2020-05-24T13:47:50.937 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:50.979 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:50.980 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:50.980 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:50.981 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:50.981 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:51.619 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:51.946 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:47:51.947 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:51.947 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:51.948 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:51.980 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:51.981 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-kubernetes.noarch 0:11.0.0-2.el7 will be installed 2020-05-24T13:47:51.983 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:51.984 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:51.985 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:51.986 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:51.987 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:51.987 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2020-05-24T13:47:51.988 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2020-05-24T13:47:51.988 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2020-05-24T13:47:51.988 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:51.989 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:51.990 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:51.991 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2020-05-24T13:47:51.991 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2020-05-24T13:47:51.992 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:51.993 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2020-05-24T13:47:51.993 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2020-05-24T13:47:51.994 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2020-05-24T13:47:51.994 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2020-05-24T13:47:51.994 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-rsa.noarch 0:3.4.1-3.el7 will be installed 2020-05-24T13:47:51.995 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:47:51.995 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2020-05-24T13:47:52.344 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:52.360 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:52.585 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:52.586 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:47:52.589 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:52.590 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:52.590 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2020-05-24T13:47:52.591 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:52.591 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:47:52.591 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:47:52.592 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:47:52.592 INFO:teuthology.orchestra.run.smithi193.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2020-05-24T13:47:52.592 INFO:teuthology.orchestra.run.smithi193.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2020-05-24T13:47:52.593 INFO:teuthology.orchestra.run.smithi193.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2020-05-24T13:47:52.593 INFO:teuthology.orchestra.run.smithi193.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2020-05-24T13:47:52.593 INFO:teuthology.orchestra.run.smithi193.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2020-05-24T13:47:52.594 INFO:teuthology.orchestra.run.smithi193.stdout: python2-kubernetes noarch 11.0.0-2.el7 epel 1.1 M 2020-05-24T13:47:52.594 INFO:teuthology.orchestra.run.smithi193.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2020-05-24T13:47:52.594 INFO:teuthology.orchestra.run.smithi193.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2020-05-24T13:47:52.595 INFO:teuthology.orchestra.run.smithi193.stdout: python2-requests-oauthlib 2020-05-24T13:47:52.595 INFO:teuthology.orchestra.run.smithi193.stdout: noarch 0.8.0-5.el7 base 35 k 2020-05-24T13:47:52.596 INFO:teuthology.orchestra.run.smithi193.stdout: python2-rsa noarch 3.4.1-3.el7 epel 68 k 2020-05-24T13:47:52.597 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:52.597 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:47:52.597 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:47:52.598 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+10 Dependent packages) 2020-05-24T13:47:52.598 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:47:52.598 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 1.7 M 2020-05-24T13:47:52.599 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 25 M 2020-05-24T13:47:52.599 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:47:53.078 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:53.078 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.5 MB/s | 1.7 MB 00:00 2020-05-24T13:47:53.094 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:47:53.108 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:47:53.181 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:47:53.182 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:47:53.187 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2020-05-24T13:47:53.862 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 5/5 2020-05-24T13:47:53.894 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-rsa-3.4.1-3.el7.noarch 1/11 2020-05-24T13:47:53.969 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2020-05-24T13:47:54.086 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 2/5 2020-05-24T13:47:54.187 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:54.254 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2020-05-24T13:47:54.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 4/5 2020-05-24T13:47:54.606 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2020-05-24T13:47:54.765 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2020-05-24T13:47:54.765 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:54.765 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:47:54.765 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:54.766 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:54.766 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:47:54.766 INFO:teuthology.orchestra.run.smithi019.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2020-05-24T13:47:54.766 INFO:teuthology.orchestra.run.smithi019.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2020-05-24T13:47:54.766 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:54.767 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:47:54.900 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-rook 2020-05-24T13:47:55.128 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:55.168 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:55.169 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:55.170 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:55.170 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:55.171 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:55.269 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2020-05-24T13:47:55.554 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 5/11 2020-05-24T13:47:55.842 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:55.894 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 6/11 2020-05-24T13:47:56.182 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:47:56.182 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:56.183 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:56.184 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:56.217 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:56.218 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-kubernetes.noarch 0:11.0.0-2.el7 will be installed 2020-05-24T13:47:56.220 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:56.221 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:56.222 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:56.223 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:56.224 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:56.224 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2020-05-24T13:47:56.225 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2020-05-24T13:47:56.225 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2020-05-24T13:47:56.226 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:56.226 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:56.227 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:56.228 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2020-05-24T13:47:56.229 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2020-05-24T13:47:56.230 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:56.230 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2020-05-24T13:47:56.230 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2020-05-24T13:47:56.231 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2020-05-24T13:47:56.231 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2020-05-24T13:47:56.232 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-rsa.noarch 0:3.4.1-3.el7 will be installed 2020-05-24T13:47:56.232 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:47:56.232 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2020-05-24T13:47:56.238 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 7/11 2020-05-24T13:47:56.544 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2020-05-24T13:47:56.623 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:56.855 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:56.855 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:47:56.857 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:56.857 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:56.857 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2020-05-24T13:47:56.858 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:56.858 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:47:56.858 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:47:56.858 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:47:56.858 INFO:teuthology.orchestra.run.smithi019.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2020-05-24T13:47:56.859 INFO:teuthology.orchestra.run.smithi019.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2020-05-24T13:47:56.859 INFO:teuthology.orchestra.run.smithi019.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2020-05-24T13:47:56.859 INFO:teuthology.orchestra.run.smithi019.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2020-05-24T13:47:56.859 INFO:teuthology.orchestra.run.smithi019.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2020-05-24T13:47:56.860 INFO:teuthology.orchestra.run.smithi019.stdout: python2-kubernetes noarch 11.0.0-2.el7 epel 1.1 M 2020-05-24T13:47:56.860 INFO:teuthology.orchestra.run.smithi019.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2020-05-24T13:47:56.861 INFO:teuthology.orchestra.run.smithi019.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2020-05-24T13:47:56.861 INFO:teuthology.orchestra.run.smithi019.stdout: python2-requests-oauthlib 2020-05-24T13:47:56.861 INFO:teuthology.orchestra.run.smithi019.stdout: noarch 0.8.0-5.el7 base 35 k 2020-05-24T13:47:56.861 INFO:teuthology.orchestra.run.smithi019.stdout: python2-rsa noarch 3.4.1-3.el7 epel 68 k 2020-05-24T13:47:56.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:56.862 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:47:56.862 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:47:56.862 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+10 Dependent packages) 2020-05-24T13:47:56.862 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:47:56.864 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 1.7 M 2020-05-24T13:47:56.864 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 25 M 2020-05-24T13:47:56.865 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:47:57.439 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:57.440 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.9 MB/s | 1.7 MB 00:00 2020-05-24T13:47:57.457 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:47:57.472 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:47:57.549 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:47:57.550 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:47:57.871 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2020-05-24T13:47:58.220 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-kubernetes-11.0.0-2.el7.noarch 10/11 2020-05-24T13:47:58.523 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-rsa-3.4.1-3.el7.noarch 1/11 2020-05-24T13:47:58.548 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 11/11 2020-05-24T13:47:58.656 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2020-05-24T13:47:58.765 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2020-05-24T13:47:58.849 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 3/11 2020-05-24T13:47:58.933 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2020-05-24T13:47:58.949 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2020-05-24T13:47:59.041 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2020-05-24T13:47:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2020-05-24T13:47:59.218 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2020-05-24T13:47:59.275 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2020-05-24T13:47:59.326 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 8/11 2020-05-24T13:47:59.435 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2020-05-24T13:47:59.544 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-rsa-3.4.1-3.el7.noarch 10/11 2020-05-24T13:47:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2020-05-24T13:47:59.872 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 5/11 2020-05-24T13:48:00.091 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-kubernetes-11.0.0-2.el7.noarch 11/11 2020-05-24T13:48:00.091 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:00.091 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:48:00.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:00.092 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:00.092 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:48:00.092 INFO:teuthology.orchestra.run.smithi193.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2020-05-24T13:48:00.092 INFO:teuthology.orchestra.run.smithi193.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2020-05-24T13:48:00.093 INFO:teuthology.orchestra.run.smithi193.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2020-05-24T13:48:00.093 INFO:teuthology.orchestra.run.smithi193.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2020-05-24T13:48:00.093 INFO:teuthology.orchestra.run.smithi193.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2020-05-24T13:48:00.093 INFO:teuthology.orchestra.run.smithi193.stdout: python2-kubernetes.noarch 0:11.0.0-2.el7 2020-05-24T13:48:00.093 INFO:teuthology.orchestra.run.smithi193.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2020-05-24T13:48:00.094 INFO:teuthology.orchestra.run.smithi193.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2020-05-24T13:48:00.094 INFO:teuthology.orchestra.run.smithi193.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2020-05-24T13:48:00.094 INFO:teuthology.orchestra.run.smithi193.stdout: python2-rsa.noarch 0:3.4.1-3.el7 2020-05-24T13:48:00.094 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:00.095 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:48:00.147 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 6/11 2020-05-24T13:48:00.225 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-ssh 2020-05-24T13:48:00.494 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:00.501 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 7/11 2020-05-24T13:48:00.539 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:00.540 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:00.541 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:00.541 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:00.541 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:00.804 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2020-05-24T13:48:01.186 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:01.527 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:48:01.527 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:01.528 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:01.529 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:48:01.561 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:01.561 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-remoto.noarch 0:1.0.0-1.el7 will be installed 2020-05-24T13:48:01.562 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python2-execnet >= 1.2.0 for package: python2-remoto-1.0.0-1.el7.noarch 2020-05-24T13:48:01.563 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:01.563 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-execnet.noarch 0:1.7.1-3.el7 will be installed 2020-05-24T13:48:01.563 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: python-apipkg for package: python-execnet-1.7.1-3.el7.noarch 2020-05-24T13:48:01.564 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:01.564 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python2-apipkg.noarch 0:1.5-4.el7 will be installed 2020-05-24T13:48:01.931 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2020-05-24T13:48:02.153 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:02.153 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:48:02.154 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:02.154 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:02.155 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2020-05-24T13:48:02.155 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:02.155 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:48:02.155 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-ssh noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:48:02.155 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:48:02.156 INFO:teuthology.orchestra.run.smithi193.stdout: python-execnet noarch 1.7.1-3.el7 epel 260 k 2020-05-24T13:48:02.156 INFO:teuthology.orchestra.run.smithi193.stdout: python2-apipkg noarch 1.5-4.el7 epel 17 k 2020-05-24T13:48:02.156 INFO:teuthology.orchestra.run.smithi193.stdout: python2-remoto noarch 1.0.0-1.el7 epel 42 k 2020-05-24T13:48:02.156 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:02.156 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:48:02.157 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:02.157 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:48:02.157 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:02.157 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 340 k 2020-05-24T13:48:02.157 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 1.1 M 2020-05-24T13:48:02.158 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:48:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-kubernetes-11.0.0-2.el7.noarch 10/11 2020-05-24T13:48:02.491 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:48:02.492 INFO:teuthology.orchestra.run.smithi193.stdout:Total 1.0 MB/s | 340 kB 00:00 2020-05-24T13:48:02.495 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:48:02.502 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:48:02.544 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:48:02.545 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:48:02.773 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 11/11 2020-05-24T13:48:02.890 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2020-05-24T13:48:02.990 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2020-05-24T13:48:03.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 3/11 2020-05-24T13:48:03.275 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-apipkg-1.5-4.el7.noarch 1/4 2020-05-24T13:48:03.369 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2020-05-24T13:48:03.467 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2020-05-24T13:48:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2020-05-24T13:48:03.657 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python-execnet-1.7.1-3.el7.noarch 2/4 2020-05-24T13:48:03.685 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2020-05-24T13:48:03.811 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 8/11 2020-05-24T13:48:03.911 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2020-05-24T13:48:04.025 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-rsa-3.4.1-3.el7.noarch 10/11 2020-05-24T13:48:04.026 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python2-remoto-1.0.0-1.el7.noarch 3/4 2020-05-24T13:48:04.326 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:48:04.431 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-kubernetes-11.0.0-2.el7.noarch 11/11 2020-05-24T13:48:04.431 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:04.432 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:48:04.432 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:04.432 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:04.432 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:48:04.432 INFO:teuthology.orchestra.run.smithi019.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2020-05-24T13:48:04.433 INFO:teuthology.orchestra.run.smithi019.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2020-05-24T13:48:04.433 INFO:teuthology.orchestra.run.smithi019.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2020-05-24T13:48:04.433 INFO:teuthology.orchestra.run.smithi019.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2020-05-24T13:48:04.433 INFO:teuthology.orchestra.run.smithi019.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2020-05-24T13:48:04.434 INFO:teuthology.orchestra.run.smithi019.stdout: python2-kubernetes.noarch 0:11.0.0-2.el7 2020-05-24T13:48:04.434 INFO:teuthology.orchestra.run.smithi019.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2020-05-24T13:48:04.435 INFO:teuthology.orchestra.run.smithi019.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2020-05-24T13:48:04.435 INFO:teuthology.orchestra.run.smithi019.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2020-05-24T13:48:04.435 INFO:teuthology.orchestra.run.smithi019.stdout: python2-rsa.noarch 0:3.4.1-3.el7 2020-05-24T13:48:04.436 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:04.436 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:48:04.454 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python-execnet-1.7.1-3.el7.noarch 1/4 2020-05-24T13:48:04.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-remoto-1.0.0-1.el7.noarch 2/4 2020-05-24T13:48:04.566 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-ssh 2020-05-24T13:48:04.671 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python2-apipkg-1.5-4.el7.noarch 3/4 2020-05-24T13:48:04.797 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:04.841 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:05.273 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:48:05.273 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:05.273 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:48:05.274 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:05.274 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:05.274 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:48:05.274 INFO:teuthology.orchestra.run.smithi193.stdout: python-execnet.noarch 0:1.7.1-3.el7 python2-apipkg.noarch 0:1.5-4.el7 2020-05-24T13:48:05.274 INFO:teuthology.orchestra.run.smithi193.stdout: python2-remoto.noarch 0:1.0.0-1.el7 2020-05-24T13:48:05.274 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:05.275 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:48:05.393 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-fuse 2020-05-24T13:48:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:05.610 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:05.652 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:05.653 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:05.654 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:05.654 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:05.654 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:05.843 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:48:05.844 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:05.844 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:05.845 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:48:05.878 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:05.879 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-remoto.noarch 0:1.0.0-1.el7 will be installed 2020-05-24T13:48:05.879 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python2-execnet >= 1.2.0 for package: python2-remoto-1.0.0-1.el7.noarch 2020-05-24T13:48:05.880 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:05.881 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-execnet.noarch 0:1.7.1-3.el7 will be installed 2020-05-24T13:48:05.881 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: python-apipkg for package: python-execnet-1.7.1-3.el7.noarch 2020-05-24T13:48:05.881 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:05.882 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python2-apipkg.noarch 0:1.5-4.el7 will be installed 2020-05-24T13:48:06.269 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:06.289 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:06.501 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:48:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:06.502 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2020-05-24T13:48:06.502 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:06.503 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:48:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-ssh noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:48:06.503 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:48:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python-execnet noarch 1.7.1-3.el7 epel 260 k 2020-05-24T13:48:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python2-apipkg noarch 1.5-4.el7 epel 17 k 2020-05-24T13:48:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python2-remoto noarch 1.0.0-1.el7 epel 42 k 2020-05-24T13:48:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:06.504 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:48:06.504 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:06.504 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:48:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:06.505 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 340 k 2020-05-24T13:48:06.505 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.1 M 2020-05-24T13:48:06.505 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:48:06.618 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:48:06.619 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:06.619 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:48:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:Total 832 kB/s | 340 kB 00:00 2020-05-24T13:48:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:48:06.922 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:48:06.966 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:48:06.966 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:48:06.997 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:07.184 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:07.184 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:48:07.185 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:07.185 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:07.185 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository 2020-05-24T13:48:07.185 INFO:teuthology.orchestra.run.smithi193.stdout: Size 2020-05-24T13:48:07.185 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:07.186 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:48:07.186 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 582 k 2020-05-24T13:48:07.186 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:07.186 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:48:07.186 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:07.187 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2020-05-24T13:48:07.187 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:07.187 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 582 k 2020-05-24T13:48:07.187 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 1.6 M 2020-05-24T13:48:07.187 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:48:07.688 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-apipkg-1.5-4.el7.noarch 1/4 2020-05-24T13:48:07.728 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:48:07.755 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:48:07.757 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:48:07.758 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:48:08.066 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python-execnet-1.7.1-3.el7.noarch 2/4 2020-05-24T13:48:08.402 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:08.419 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python2-remoto-1.0.0-1.el7.noarch 3/4 2020-05-24T13:48:08.701 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:48:08.812 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python-execnet-1.7.1-3.el7.noarch 1/4 2020-05-24T13:48:08.895 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-remoto-1.0.0-1.el7.noarch 2/4 2020-05-24T13:48:08.918 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:08.919 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:08.919 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:48:08.919 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:08.919 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:08.919 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:48:08.979 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python2-apipkg-1.5-4.el7.noarch 3/4 2020-05-24T13:48:09.039 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs2 2020-05-24T13:48:09.242 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:09.281 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:09.282 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:09.283 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:09.283 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:09.283 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:09.406 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:48:09.407 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:09.407 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:48:09.407 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:09.407 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:09.408 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:48:09.408 INFO:teuthology.orchestra.run.smithi019.stdout: python-execnet.noarch 0:1.7.1-3.el7 python2-apipkg.noarch 0:1.5-4.el7 2020-05-24T13:48:09.408 INFO:teuthology.orchestra.run.smithi019.stdout: python2-remoto.noarch 0:1.0.0-1.el7 2020-05-24T13:48:09.408 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:09.408 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:48:09.539 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-fuse 2020-05-24T13:48:09.743 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:09.782 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:09.782 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:09.783 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:09.783 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:09.784 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:09.915 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:10.238 INFO:teuthology.orchestra.run.smithi193.stdout:Package 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:48:10.239 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do 2020-05-24T13:48:10.300 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs-devel 2020-05-24T13:48:10.439 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:10.544 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:10.589 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:10.590 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:10.590 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:10.591 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:10.591 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:10.777 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:48:10.777 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:10.778 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:11.176 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:11.233 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:11.368 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:48:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:11.369 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository 2020-05-24T13:48:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: Size 2020-05-24T13:48:11.370 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:11.370 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:48:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 582 k 2020-05-24T13:48:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:11.371 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:48:11.371 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:11.371 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2020-05-24T13:48:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:11.372 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 582 k 2020-05-24T13:48:11.372 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.6 M 2020-05-24T13:48:11.372 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:48:11.568 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:48:11.569 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:11.569 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:11.569 INFO:teuthology.orchestra.run.smithi193.stdout:--> Processing Dependency: librados-devel = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:48:11.604 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:11.604 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:11.949 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:48:11.976 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:48:11.978 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:48:11.979 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:11.979 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:48:12.204 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:12.204 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:48:12.205 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:12.205 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:12.206 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository 2020-05-24T13:48:12.206 INFO:teuthology.orchestra.run.smithi193.stdout: Size 2020-05-24T13:48:12.206 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:12.206 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:48:12.207 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 17 k 2020-05-24T13:48:12.207 INFO:teuthology.orchestra.run.smithi193.stdout:Installing for dependencies: 2020-05-24T13:48:12.207 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 92 k 2020-05-24T13:48:12.207 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:12.207 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:48:12.208 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:12.208 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package (+1 Dependent package) 2020-05-24T13:48:12.208 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:12.208 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 109 k 2020-05-24T13:48:12.209 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 391 k 2020-05-24T13:48:12.209 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:48:12.684 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:48:12.684 INFO:teuthology.orchestra.run.smithi193.stdout:Total 227 kB/s | 109 kB 00:00 2020-05-24T13:48:12.687 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:48:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:12.711 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:48:12.714 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:48:12.714 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:48:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:13.225 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:48:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:13.226 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:48:13.308 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:48:13.351 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs2 2020-05-24T13:48:13.560 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:13.570 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:48:13.599 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:13.600 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:13.600 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:13.601 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:13.601 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:13.675 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:48:14.160 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:48:14.160 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:14.160 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:48:14.160 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:14.161 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:14.161 INFO:teuthology.orchestra.run.smithi193.stdout:Dependency Installed: 2020-05-24T13:48:14.161 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:14.161 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:14.161 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:48:14.250 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:14.281 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install librados2 2020-05-24T13:48:14.517 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:14.558 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:14.560 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:14.560 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:14.561 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:14.561 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:14.586 INFO:teuthology.orchestra.run.smithi019.stdout:Package 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:48:14.586 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do 2020-05-24T13:48:14.647 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs-devel 2020-05-24T13:48:14.837 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:14.876 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:15.200 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:15.494 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:15.533 INFO:teuthology.orchestra.run.smithi193.stdout:Package 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:48:15.533 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do 2020-05-24T13:48:15.593 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install librbd1 2020-05-24T13:48:15.829 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:48:15.829 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:15.830 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:15.830 INFO:teuthology.orchestra.run.smithi019.stdout:--> Processing Dependency: librados-devel = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:48:15.838 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:15.863 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:15.864 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:15.892 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:15.892 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:15.893 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:15.893 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:15.893 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:16.260 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:16.491 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:16.492 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:48:16.493 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:16.493 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:16.493 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository 2020-05-24T13:48:16.494 INFO:teuthology.orchestra.run.smithi019.stdout: Size 2020-05-24T13:48:16.494 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:16.494 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:48:16.494 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 17 k 2020-05-24T13:48:16.494 INFO:teuthology.orchestra.run.smithi019.stdout:Installing for dependencies: 2020-05-24T13:48:16.495 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 92 k 2020-05-24T13:48:16.495 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:16.496 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:48:16.496 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:16.496 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package (+1 Dependent package) 2020-05-24T13:48:16.496 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:16.497 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 109 k 2020-05-24T13:48:16.497 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 391 k 2020-05-24T13:48:16.497 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:48:16.525 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:16.859 INFO:teuthology.orchestra.run.smithi193.stdout:Package 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:48:16.860 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do 2020-05-24T13:48:16.923 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install python-ceph 2020-05-24T13:48:17.066 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:48:17.067 INFO:teuthology.orchestra.run.smithi019.stdout:Total 190 kB/s | 109 kB 00:00 2020-05-24T13:48:17.073 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:48:17.094 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:48:17.096 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:48:17.097 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:48:17.121 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:17.151 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:17.152 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:17.152 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:17.153 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:17.153 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:17.656 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:48:17.734 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:48:18.057 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:48:18.058 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:18.058 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:18.065 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:48:18.429 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:18.566 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:48:18.566 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:18.566 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:48:18.567 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:18.567 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:18.567 INFO:teuthology.orchestra.run.smithi019.stdout:Dependency Installed: 2020-05-24T13:48:18.567 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:18.567 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:18.568 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:48:18.614 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:18.614 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:48:18.615 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:18.616 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:18.616 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository 2020-05-24T13:48:18.616 INFO:teuthology.orchestra.run.smithi193.stdout: Size 2020-05-24T13:48:18.617 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:18.617 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:48:18.617 INFO:teuthology.orchestra.run.smithi193.stdout: python-ceph-compat x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.2 k 2020-05-24T13:48:18.618 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:18.618 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:48:18.618 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:18.618 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2020-05-24T13:48:18.619 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:18.619 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 2.2 k 2020-05-24T13:48:18.619 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 0 2020-05-24T13:48:18.620 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:48:18.703 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install librados2 2020-05-24T13:48:18.940 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:18.947 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:48:18.948 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:48:18.950 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:48:18.951 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:48:18.982 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:18.983 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:18.984 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:18.984 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:18.984 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:19.528 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:19.656 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:19.998 INFO:teuthology.orchestra.run.smithi019.stdout:Package 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:48:19.998 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do 2020-05-24T13:48:20.063 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install librbd1 2020-05-24T13:48:20.237 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:20.274 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:20.275 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:20.275 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:20.276 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:20.276 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:20.765 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:20.766 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:20.766 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:48:20.766 INFO:teuthology.orchestra.run.smithi193.stdout: python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:20.767 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:20.767 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:48:20.876 INFO:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-fuse 2020-05-24T13:48:20.896 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:21.115 INFO:teuthology.orchestra.run.smithi193.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:21.156 INFO:teuthology.orchestra.run.smithi193.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:21.156 INFO:teuthology.orchestra.run.smithi193.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:21.157 INFO:teuthology.orchestra.run.smithi193.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:21.158 INFO:teuthology.orchestra.run.smithi193.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:21.158 INFO:teuthology.orchestra.run.smithi193.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:21.236 INFO:teuthology.orchestra.run.smithi019.stdout:Package 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:48:21.236 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do 2020-05-24T13:48:21.295 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install python-ceph 2020-05-24T13:48:21.512 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:21.552 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:21.795 INFO:teuthology.orchestra.run.smithi193.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:22.123 INFO:teuthology.orchestra.run.smithi193.stdout:Resolving Dependencies 2020-05-24T13:48:22.124 INFO:teuthology.orchestra.run.smithi193.stdout:--> Running transaction check 2020-05-24T13:48:22.124 INFO:teuthology.orchestra.run.smithi193.stdout:---> Package rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:22.170 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:22.501 INFO:teuthology.orchestra.run.smithi193.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:22.508 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:48:22.508 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:22.509 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:22.687 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:22.687 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies Resolved 2020-05-24T13:48:22.688 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:22.689 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:22.689 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository 2020-05-24T13:48:22.689 INFO:teuthology.orchestra.run.smithi193.stdout: Size 2020-05-24T13:48:22.689 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:22.689 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2020-05-24T13:48:22.690 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 75 k 2020-05-24T13:48:22.690 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:22.691 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2020-05-24T13:48:22.691 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2020-05-24T13:48:22.691 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2020-05-24T13:48:22.691 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:22.691 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 75 k 2020-05-24T13:48:22.691 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 196 k 2020-05-24T13:48:22.692 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading packages: 2020-05-24T13:48:22.905 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:23.097 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:48:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:23.098 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository 2020-05-24T13:48:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: Size 2020-05-24T13:48:23.099 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:23.099 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:48:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: python-ceph-compat x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.2 k 2020-05-24T13:48:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:23.100 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:48:23.100 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:23.100 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2020-05-24T13:48:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:23.101 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.2 k 2020-05-24T13:48:23.101 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2020-05-24T13:48:23.101 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:48:23.112 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2020-05-24T13:48:23.158 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2020-05-24T13:48:23.158 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded 2020-05-24T13:48:23.159 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2020-05-24T13:48:23.484 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:48:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:48:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:48:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:48:23.777 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:24.269 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:24.269 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:24.269 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2020-05-24T13:48:24.270 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:24.270 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:24.270 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2020-05-24T13:48:24.381 DEBUG:teuthology.parallel:result is None 2020-05-24T13:48:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:24.930 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:24.931 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:48:24.931 INFO:teuthology.orchestra.run.smithi019.stdout: python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:24.946 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:24.946 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:48:25.042 INFO:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-fuse 2020-05-24T13:48:25.235 INFO:teuthology.orchestra.run.smithi019.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:25.271 INFO:teuthology.orchestra.run.smithi019.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:48:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:25.931 INFO:teuthology.orchestra.run.smithi019.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:26.268 INFO:teuthology.orchestra.run.smithi019.stdout:Resolving Dependencies 2020-05-24T13:48:26.269 INFO:teuthology.orchestra.run.smithi019.stdout:--> Running transaction check 2020-05-24T13:48:26.270 INFO:teuthology.orchestra.run.smithi019.stdout:---> Package rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:26.672 INFO:teuthology.orchestra.run.smithi019.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:26.863 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:26.864 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies Resolved 2020-05-24T13:48:26.865 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:26.865 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:26.865 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository 2020-05-24T13:48:26.865 INFO:teuthology.orchestra.run.smithi019.stdout: Size 2020-05-24T13:48:26.865 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:26.866 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2020-05-24T13:48:26.866 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 75 k 2020-05-24T13:48:26.866 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:26.866 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2020-05-24T13:48:26.866 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2020-05-24T13:48:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2020-05-24T13:48:26.867 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 75 k 2020-05-24T13:48:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 196 k 2020-05-24T13:48:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading packages: 2020-05-24T13:48:27.399 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2020-05-24T13:48:27.425 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2020-05-24T13:48:27.427 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded 2020-05-24T13:48:27.428 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2020-05-24T13:48:28.067 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:28.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:28.600 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:28.601 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2020-05-24T13:48:28.601 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:28.601 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:28.601 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2020-05-24T13:48:28.715 DEBUG:teuthology.parallel:result is None 2020-05-24T13:48:28.716 INFO:teuthology.orchestra.run.smithi019:> sudo lsb_release -is 2020-05-24T13:48:28.867 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS 2020-05-24T13:48:28.868 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:48:28.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:48:29.126 INFO:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-05-24T13:48:29.269 INFO:teuthology.orchestra.run.smithi019.stdout:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:29.271 INFO:teuthology.packaging:The installed version of ceph is 14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:29.271 INFO:teuthology.task.install:The correct ceph version 14.2.9-537.g6fe217b2dd is installed. 2020-05-24T13:48:29.272 INFO:teuthology.orchestra.run.smithi193:> sudo lsb_release -is 2020-05-24T13:48:29.420 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS 2020-05-24T13:48:29.423 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:48:29.424 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:48:29.682 INFO:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-05-24T13:48:29.811 INFO:teuthology.orchestra.run.smithi193.stdout:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:29.812 INFO:teuthology.packaging:The installed version of ceph is 14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:29.812 INFO:teuthology.task.install:The correct ceph version 14.2.9-537.g6fe217b2dd is installed. 2020-05-24T13:48:29.813 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-05-24T13:48:29.813 INFO:teuthology.orchestra.run.smithi019:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-05-24T13:48:29.938 INFO:teuthology.orchestra.run.smithi193:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-05-24T13:48:30.061 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-05-24T13:48:30.062 INFO:teuthology.orchestra.run.smithi019:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-05-24T13:48:30.186 INFO:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-05-24T13:48:30.306 INFO:teuthology.orchestra.run.smithi193:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-05-24T13:48:30.425 INFO:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-05-24T13:48:30.531 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-05-24T13:48:30.531 INFO:teuthology.orchestra.run.smithi019:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-05-24T13:48:30.656 INFO:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-05-24T13:48:30.763 INFO:teuthology.orchestra.run.smithi193:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-05-24T13:48:30.880 INFO:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-05-24T13:48:30.981 INFO:teuthology.run_tasks:Running task ceph... 2020-05-24T13:48:31.052 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-05-24T13:48:31.053 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 777 /var/log/ceph 2020-05-24T13:48:31.057 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 777 /var/log/ceph 2020-05-24T13:48:31.175 INFO:tasks.ceph:Disabling ceph logrotate... 2020-05-24T13:48:31.176 INFO:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/logrotate.d/ceph 2020-05-24T13:48:31.181 INFO:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/logrotate.d/ceph 2020-05-24T13:48:31.292 INFO:tasks.ceph:Creating extra log directories... 2020-05-24T13:48:31.293 INFO:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-05-24T13:48:31.297 INFO:teuthology.orchestra.run.smithi193:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-05-24T13:48:31.413 INFO:tasks.ceph:Creating ceph cluster c1... 2020-05-24T13:48:31.413 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'c1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-05-24T13:48:31.414 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087418', 'branch': 'nautilus', 'description': 'rgw/multisite/{clusters.yaml frontend/beast.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml}', 'email': 'tchaikov@gmail.com', 'first_in_suite': False, 'job_id': '5087418', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi', 'nuke-on-error': True, 'os_type': 'centos', 'overrides': {'admin_socket': {'branch': 'nautilus'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c1.client.1'], 'name': 'test-zone3'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone4'}]}]}, 'workunit': {'branch': 'nautilus-py3', 'sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff'}}, 'owner': 'scheduled_kchai@teuthology', 'priority': 90, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'nautilus-py3', 'suite_path': '/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/tchaikov/ceph.git', 'suite_sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2'}}, {'rgw': {'c1.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c1.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.27836', 'targets': {'smithi019.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnzKzvSvSSWhp8CQJE68ha/aIH8OK1pi/nONXFErfC78oXKR8ZNIhgv5FDzind/CpyduLhpPcJnw+zH2e3jwHhi0e6YEGVQEIBL8pr0+Sf0fZ9+jp4nBWKF0ZTbmy2hIidWRC4S7UCmFKYiZuyTa+FKyV0YcMB0ZRMa21Y73JVd3q01kJhd9i7F0PkutoM0SzMDC9xTmXxsH60VFXIs+um4giQplsiL5QZftzbKyyDw9ayPtMUOhR9AeI7YfmETAymVal3Xy5mzAPJIhmc0lPyQoDwYz1A2+e0VKUtb56uvbKvUBLS5qdoUZhxTLGhR80urGGYmCAeY+kza7R9eVkP', 'smithi193.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZVvdI2EmeM5Uc9ruLPP62O+Nj5/XERWhHEup3D2vviYzaLHVDcmOrC9OzuCxuSIi9SWsopYQUWD3ZekNEhKP0nqg2LVosDApxzBJ7c6gdkDMrwx6bm2uDVZVt/f6ghtUjnScO9SQeGFwpa5lkp3sxBBSkymUBAFklUc5qHzYdYpa3GhVtk7aeBOHN1de5nra+K2r94YhtOeSpXXKZybWPPenBWzRXZ8EoAkbgJPistYN1TAtCs6/54caTlA814tZ5akAMXihxO+qtUgdomjlJvasVtzTdrFHBEVhnQzt9Fd68/P9MicuvkffceblmK/n4xEjvWJttOVMAubyjtRvh'}} 2020-05-24T13:48:31.415 INFO:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2020-05-24T13:48:31.418 INFO:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2020-05-24T13:48:31.503 INFO:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:48:31.507 INFO:teuthology.orchestra.run.smithi193:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:48:31.711 DEBUG:teuthology.orchestra.remote:smithi019:/scratch_devs is 72B 2020-05-24T13:48:31.801 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-05-24T13:48:31.802 INFO:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-05-24T13:48:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-0’ 2020-05-24T13:48:31.881 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:31.882 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 104678 Links: 1 2020-05-24T13:48:31.882 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:31.882 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:31.882 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2020-05-24 13:47:30.460423893 +0000 2020-05-24T13:48:31.882 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2020-05-24 13:45:49.759721795 +0000 2020-05-24T13:48:31.882 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2020-05-24 13:45:49.759721795 +0000 2020-05-24T13:48:31.883 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2020-05-24T13:48:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2020-05-24T13:48:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2020-05-24T13:48:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes (512 B) copied, 0.000439801 s, 1.2 MB/s 2020-05-24T13:48:31.915 INFO:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-05-24T13:48:31.996 INFO:teuthology.orchestra.run.smithi019.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-1’ 2020-05-24T13:48:31.996 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:31.997 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 112716 Links: 1 2020-05-24T13:48:31.997 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:31.997 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:31.997 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2020-05-24 13:47:30.460423893 +0000 2020-05-24T13:48:31.997 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2020-05-24 13:45:50.260717469 +0000 2020-05-24T13:48:31.998 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2020-05-24 13:45:50.260717469 +0000 2020-05-24T13:48:31.998 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2020-05-24T13:48:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2020-05-24T13:48:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2020-05-24T13:48:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes (512 B) copied, 0.00036305 s, 1.4 MB/s 2020-05-24T13:48:32.029 INFO:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-05-24T13:48:32.097 INFO:teuthology.orchestra.run.smithi019.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2020-05-24T13:48:32.098 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 104430 Links: 1 2020-05-24T13:48:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:32.099 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2020-05-24 13:47:30.460423893 +0000 2020-05-24T13:48:32.099 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2020-05-24 13:45:50.745713281 +0000 2020-05-24T13:48:32.099 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2020-05-24 13:45:50.745713281 +0000 2020-05-24T13:48:32.099 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2020-05-24T13:48:32.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2020-05-24T13:48:32.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2020-05-24T13:48:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes (512 B) copied, 0.000315119 s, 1.6 MB/s 2020-05-24T13:48:32.126 INFO:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-05-24T13:48:32.187 INFO:teuthology.orchestra.run.smithi019.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-3’ 2020-05-24T13:48:32.187 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:32.188 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 109912 Links: 1 2020-05-24T13:48:32.188 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:32.188 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:32.188 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2020-05-24 13:47:30.460423893 +0000 2020-05-24T13:48:32.188 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2020-05-24 13:45:51.206709300 +0000 2020-05-24T13:48:32.189 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2020-05-24 13:45:51.206709300 +0000 2020-05-24T13:48:32.189 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2020-05-24T13:48:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2020-05-24T13:48:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2020-05-24T13:48:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes (512 B) copied, 0.000353502 s, 1.4 MB/s 2020-05-24T13:48:32.212 INFO:tasks.ceph:osd dev map: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:32.212 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:48:32.213 INFO:tasks.ceph:Generating config... 2020-05-24T13:48:32.214 INFO:tasks.ceph:[client] debug rgw = 20 2020-05-24T13:48:32.215 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2020-05-24T13:48:32.215 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:48:32.215 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2020-05-24T13:48:32.216 INFO:tasks.ceph:[client] rgw data log num shards = 4 2020-05-24T13:48:32.216 INFO:tasks.ceph:[client] rgw md log max shards = 4 2020-05-24T13:48:32.216 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2020-05-24T13:48:32.216 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2020-05-24T13:48:32.217 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-05-24T13:48:32.217 INFO:tasks.ceph:[mgr] debug ms = 1 2020-05-24T13:48:32.217 INFO:tasks.ceph:[mon] debug mon = 20 2020-05-24T13:48:32.217 INFO:tasks.ceph:[mon] debug ms = 1 2020-05-24T13:48:32.218 INFO:tasks.ceph:[mon] debug paxos = 20 2020-05-24T13:48:32.218 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2020-05-24T13:48:32.218 INFO:tasks.ceph:[osd] debug filestore = 20 2020-05-24T13:48:32.219 INFO:tasks.ceph:[osd] debug journal = 20 2020-05-24T13:48:32.219 INFO:tasks.ceph:[osd] debug ms = 20 2020-05-24T13:48:32.219 INFO:tasks.ceph:[osd] debug osd = 25 2020-05-24T13:48:32.219 INFO:tasks.ceph:[osd] osd fast shutdown = False 2020-05-24T13:48:32.220 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2020-05-24T13:48:32.220 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2020-05-24T13:48:32.220 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2020-05-24T13:48:32.221 INFO:tasks.ceph:Setting up c1.mon.a... 2020-05-24T13:48:32.221 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c1.keyring 2020-05-24T13:48:32.379 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/c1.keyring 2020-05-24T13:48:32.392 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c1.keyring 2020-05-24T13:48:32.558 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/c1.keyring 2020-05-24T13:48:32.674 DEBUG:tasks.ceph:Ceph mon addresses: [('c1.mon.a', '172.21.15.19')] 2020-05-24T13:48:32.675 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.19 --print /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:32.843 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:32.843 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: generated fsid d75753f9-1100-4c74-b8ad-924f57b3114a 2020-05-24T13:48:32.844 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 0 2020-05-24T13:48:32.844 INFO:teuthology.orchestra.run.smithi019.stdout:fsid d75753f9-1100-4c74-b8ad-924f57b3114a 2020-05-24T13:48:32.844 INFO:teuthology.orchestra.run.smithi019.stdout:last_changed 2020-05-24 13:48:32.842342 2020-05-24T13:48:32.844 INFO:teuthology.orchestra.run.smithi019.stdout:created 2020-05-24 13:48:32.842342 2020-05-24T13:48:32.844 INFO:teuthology.orchestra.run.smithi019.stdout:min_mon_release 0 (unknown) 2020-05-24T13:48:32.845 INFO:teuthology.orchestra.run.smithi019.stdout:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2020-05-24T13:48:32.845 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c1.monmap (1 monitors) 2020-05-24T13:48:32.846 INFO:tasks.ceph:Writing /etc/ceph/c1.conf for FSID d75753f9-1100-4c74-b8ad-924f57b3114a... 2020-05-24T13:48:32.847 INFO:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2020-05-24T13:48:32.858 INFO:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2020-05-24T13:48:33.020 INFO:teuthology.orchestra.run.smithi193.stdout:[global] 2020-05-24T13:48:33.020 INFO:teuthology.orchestra.run.smithi193.stdout: chdir = "" 2020-05-24T13:48:33.020 INFO:teuthology.orchestra.run.smithi193.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:48:33.020 INFO:teuthology.orchestra.run.smithi193.stdout: auth supported = cephx 2020-05-24T13:48:33.021 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.021 INFO:teuthology.orchestra.run.smithi193.stdout: filestore xattr use omap = true 2020-05-24T13:48:33.021 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.021 INFO:teuthology.orchestra.run.smithi193.stdout: mon clock drift allowed = 1.000 2020-05-24T13:48:33.021 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.022 INFO:teuthology.orchestra.run.smithi193.stdout: osd crush chooseleaf type = 0 2020-05-24T13:48:33.022 INFO:teuthology.orchestra.run.smithi193.stdout: auth debug = true 2020-05-24T13:48:33.022 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.022 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on old message = true 2020-05-24T13:48:33.022 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on bug = true 2020-05-24T13:48:33.023 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.023 INFO:teuthology.orchestra.run.smithi193.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:48:33.023 INFO:teuthology.orchestra.run.smithi193.stdout: mon pg warn max object skew = 0 2020-05-24T13:48:33.023 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.023 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default size = 2 2020-05-24T13:48:33.023 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.024 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow primary affinity = true 2020-05-24T13:48:33.024 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow pg remap = true 2020-05-24T13:48:33.024 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:48:33.024 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:48:33.024 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on no sortbitwise = false 2020-05-24T13:48:33.025 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:48:33.025 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on too few osds = false 2020-05-24T13:48:33.025 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:48:33.025 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:48:33.025 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.025 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:48:33.026 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.026 INFO:teuthology.orchestra.run.smithi193.stdout: osd default data pool replay window = 5 2020-05-24T13:48:33.026 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.026 INFO:teuthology.orchestra.run.smithi193.stdout: mon allow pool delete = true 2020-05-24T13:48:33.026 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.027 INFO:teuthology.orchestra.run.smithi193.stdout: mon cluster log file level = debug 2020-05-24T13:48:33.027 INFO:teuthology.orchestra.run.smithi193.stdout: debug asserts on shutdown = true 2020-05-24T13:48:33.027 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.027 INFO:teuthology.orchestra.run.smithi193.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:48:33.027 INFO:teuthology.orchestra.run.smithi193.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:48:33.027 INFO:teuthology.orchestra.run.smithi193.stdout: mon host = 172.21.15.19 2020-05-24T13:48:33.028 INFO:teuthology.orchestra.run.smithi193.stdout: osd heartbeat grace = 40 2020-05-24T13:48:33.028 INFO:teuthology.orchestra.run.smithi193.stdout: fsid = d75753f9-1100-4c74-b8ad-924f57b3114a 2020-05-24T13:48:33.028 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.028 INFO:teuthology.orchestra.run.smithi193.stdout:[osd] 2020-05-24T13:48:33.028 INFO:teuthology.orchestra.run.smithi193.stdout: osd journal size = 100 2020-05-24T13:48:33.029 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.029 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub load threshold = 5.0 2020-05-24T13:48:33.029 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub max interval = 600 2020-05-24T13:48:33.029 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.030 INFO:teuthology.orchestra.run.smithi193.stdout: osd recover clone overlap = true 2020-05-24T13:48:33.030 INFO:teuthology.orchestra.run.smithi193.stdout: osd recovery max chunk = 1048576 2020-05-24T13:48:33.030 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.030 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug shutdown = true 2020-05-24T13:48:33.030 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug op order = true 2020-05-24T13:48:33.031 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug verify stray on activate = true 2020-05-24T13:48:33.031 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.031 INFO:teuthology.orchestra.run.smithi193.stdout: osd open classes on start = true 2020-05-24T13:48:33.031 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug pg log writeout = true 2020-05-24T13:48:33.031 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.032 INFO:teuthology.orchestra.run.smithi193.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:48:33.032 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.032 INFO:teuthology.orchestra.run.smithi193.stdout: osd map max advance = 10 2020-05-24T13:48:33.032 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.033 INFO:teuthology.orchestra.run.smithi193.stdout: journal zero on create = true 2020-05-24T13:48:33.033 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.033 INFO:teuthology.orchestra.run.smithi193.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:48:33.033 INFO:teuthology.orchestra.run.smithi193.stdout: filestore apply finisher threads = 3 2020-05-24T13:48:33.033 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.034 INFO:teuthology.orchestra.run.smithi193.stdout: bdev debug aio = true 2020-05-24T13:48:33.034 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug misdirected ops = true 2020-05-24T13:48:33.034 INFO:teuthology.orchestra.run.smithi193.stdout: debug filestore = 20 2020-05-24T13:48:33.034 INFO:teuthology.orchestra.run.smithi193.stdout: debug journal = 20 2020-05-24T13:48:33.034 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 20 2020-05-24T13:48:33.034 INFO:teuthology.orchestra.run.smithi193.stdout: debug osd = 25 2020-05-24T13:48:33.035 INFO:teuthology.orchestra.run.smithi193.stdout: osd fast shutdown = False 2020-05-24T13:48:33.035 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.035 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr] 2020-05-24T13:48:33.035 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2020-05-24T13:48:33.035 INFO:teuthology.orchestra.run.smithi193.stdout: debug mgr = 20 2020-05-24T13:48:33.036 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2020-05-24T13:48:33.036 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2020-05-24T13:48:33.036 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:33.036 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:33.036 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.036 INFO:teuthology.orchestra.run.smithi193.stdout:[mon] 2020-05-24T13:48:33.037 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2020-05-24T13:48:33.037 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2020-05-24T13:48:33.037 INFO:teuthology.orchestra.run.smithi193.stdout: debug paxos = 20 2020-05-24T13:48:33.037 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2020-05-24T13:48:33.037 INFO:teuthology.orchestra.run.smithi193.stdout: mon data avail warn = 5 2020-05-24T13:48:33.038 INFO:teuthology.orchestra.run.smithi193.stdout: mon mgr mkfs grace = 240 2020-05-24T13:48:33.038 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:33.038 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd reporter subtree level = osd 2020-05-24T13:48:33.038 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd prime pg temp = true 2020-05-24T13:48:33.038 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:33.038 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd crush smoke test = False 2020-05-24T13:48:33.039 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:48:33.039 INFO:teuthology.orchestra.run.smithi193.stdout:[client] 2020-05-24T13:48:33.039 INFO:teuthology.orchestra.run.smithi193.stdout: rgw cache enabled = true 2020-05-24T13:48:33.039 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable ops log = true 2020-05-24T13:48:33.039 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable usage log = true 2020-05-24T13:48:33.040 INFO:teuthology.orchestra.run.smithi193.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:48:33.040 INFO:teuthology.orchestra.run.smithi193.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:48:33.040 INFO:teuthology.orchestra.run.smithi193.stdout: debug rgw = 20 2020-05-24T13:48:33.040 INFO:teuthology.orchestra.run.smithi193.stdout: rgw crypt require ssl = False 2020-05-24T13:48:33.041 INFO:teuthology.orchestra.run.smithi193.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:48:33.041 INFO:teuthology.orchestra.run.smithi193.stdout: rgw curl low speed time = 300 2020-05-24T13:48:33.041 INFO:teuthology.orchestra.run.smithi193.stdout: rgw data log num shards = 4 2020-05-24T13:48:33.041 INFO:teuthology.orchestra.run.smithi193.stdout: rgw md log max shards = 4 2020-05-24T13:48:33.043 INFO:teuthology.orchestra.run.smithi193.stdout: rgw sync log trim interval = 0 2020-05-24T13:48:33.043 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.a] 2020-05-24T13:48:33.043 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.0] 2020-05-24T13:48:33.043 INFO:teuthology.orchestra.run.smithi193.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:48:33.044 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.1] 2020-05-24T13:48:33.044 INFO:teuthology.orchestra.run.smithi193.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:48:33.044 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.2] 2020-05-24T13:48:33.044 INFO:teuthology.orchestra.run.smithi193.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:48:33.046 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2020-05-24T13:48:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2020-05-24T13:48:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:48:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2020-05-24T13:48:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2020-05-24T13:48:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2020-05-24T13:48:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2020-05-24T13:48:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2020-05-24T13:48:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2020-05-24T13:48:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2020-05-24T13:48:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:48:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2020-05-24T13:48:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2020-05-24T13:48:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2020-05-24T13:48:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2020-05-24T13:48:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:48:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:48:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2020-05-24T13:48:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:48:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2020-05-24T13:48:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:48:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:48:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:48:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2020-05-24T13:48:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2020-05-24T13:48:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2020-05-24T13:48:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2020-05-24T13:48:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:48:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:48:33.065 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = 172.21.15.19 2020-05-24T13:48:33.065 INFO:teuthology.orchestra.run.smithi019.stdout: osd heartbeat grace = 40 2020-05-24T13:48:33.065 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = d75753f9-1100-4c74-b8ad-924f57b3114a 2020-05-24T13:48:33.065 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.065 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2020-05-24T13:48:33.065 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2020-05-24T13:48:33.066 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.066 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2020-05-24T13:48:33.066 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2020-05-24T13:48:33.066 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.066 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2020-05-24T13:48:33.067 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2020-05-24T13:48:33.067 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.067 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2020-05-24T13:48:33.067 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2020-05-24T13:48:33.067 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2020-05-24T13:48:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2020-05-24T13:48:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2020-05-24T13:48:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:48:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.069 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2020-05-24T13:48:33.069 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.069 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2020-05-24T13:48:33.069 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.069 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:48:33.070 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2020-05-24T13:48:33.070 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.070 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2020-05-24T13:48:33.070 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2020-05-24T13:48:33.070 INFO:teuthology.orchestra.run.smithi019.stdout: debug filestore = 20 2020-05-24T13:48:33.070 INFO:teuthology.orchestra.run.smithi019.stdout: debug journal = 20 2020-05-24T13:48:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 20 2020-05-24T13:48:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 25 2020-05-24T13:48:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: osd fast shutdown = False 2020-05-24T13:48:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2020-05-24T13:48:33.072 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2020-05-24T13:48:33.072 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2020-05-24T13:48:33.072 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2020-05-24T13:48:33.072 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2020-05-24T13:48:33.072 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:33.072 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:33.073 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.073 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2020-05-24T13:48:33.073 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2020-05-24T13:48:33.073 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2020-05-24T13:48:33.074 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2020-05-24T13:48:33.074 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2020-05-24T13:48:33.074 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2020-05-24T13:48:33.074 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2020-05-24T13:48:33.075 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:33.075 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2020-05-24T13:48:33.075 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2020-05-24T13:48:33.076 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:33.076 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd crush smoke test = False 2020-05-24T13:48:33.076 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:48:33.076 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2020-05-24T13:48:33.077 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2020-05-24T13:48:33.077 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2020-05-24T13:48:33.077 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2020-05-24T13:48:33.077 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:48:33.078 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:48:33.078 INFO:teuthology.orchestra.run.smithi019.stdout: debug rgw = 20 2020-05-24T13:48:33.078 INFO:teuthology.orchestra.run.smithi019.stdout: rgw crypt require ssl = False 2020-05-24T13:48:33.078 INFO:teuthology.orchestra.run.smithi019.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:48:33.079 INFO:teuthology.orchestra.run.smithi019.stdout: rgw curl low speed time = 300 2020-05-24T13:48:33.079 INFO:teuthology.orchestra.run.smithi019.stdout: rgw data log num shards = 4 2020-05-24T13:48:33.079 INFO:teuthology.orchestra.run.smithi019.stdout: rgw md log max shards = 4 2020-05-24T13:48:33.079 INFO:teuthology.orchestra.run.smithi019.stdout: rgw sync log trim interval = 0 2020-05-24T13:48:33.080 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2020-05-24T13:48:33.080 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2020-05-24T13:48:33.080 INFO:teuthology.orchestra.run.smithi019.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:48:33.080 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2020-05-24T13:48:33.081 INFO:teuthology.orchestra.run.smithi019.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:48:33.081 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2020-05-24T13:48:33.081 INFO:teuthology.orchestra.run.smithi019.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:48:33.083 INFO:tasks.ceph:Creating admin key on c1.mon.a... 2020-05-24T13:48:33.083 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c1.keyring 2020-05-24T13:48:33.294 INFO:tasks.ceph:Copying monmap to all nodes... 2020-05-24T13:48:33.385 DEBUG:teuthology.orchestra.remote:smithi019:/etc/ceph/c1.keyring is 206B 2020-05-24T13:48:33.586 DEBUG:teuthology.orchestra.remote:smithi019:/home/ubuntu/cephtest/c1.monmap is 177B 2020-05-24T13:48:33.680 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:48:33.686 INFO:teuthology.orchestra.run.smithi019:> sudo sh -c 'cat > /etc/ceph/c1.keyring' && sudo chmod 0644 /etc/ceph/c1.keyring 2020-05-24T13:48:33.823 INFO:teuthology.orchestra.run.smithi019:> cat > /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:33.896 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:48:33.897 INFO:teuthology.orchestra.run.smithi193:> sudo sh -c 'cat > /etc/ceph/c1.keyring' && sudo chmod 0644 /etc/ceph/c1.keyring 2020-05-24T13:48:34.043 INFO:teuthology.orchestra.run.smithi193:> cat > /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:34.121 INFO:tasks.ceph:Setting up mon nodes... 2020-05-24T13:48:34.122 INFO:tasks.ceph:Setting up mgr nodes... 2020-05-24T13:48:34.123 INFO:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mgr/c1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c1-x/keyring 2020-05-24T13:48:34.337 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mgr/c1-x/keyring 2020-05-24T13:48:34.342 INFO:tasks.ceph:Setting up mds nodes... 2020-05-24T13:48:34.343 INFO:tasks.ceph_client:Setting up client nodes... 2020-05-24T13:48:34.343 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c1.client.0.keyring && sudo chmod 0644 /etc/ceph/c1.client.0.keyring 2020-05-24T13:48:34.529 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/c1.client.0.keyring 2020-05-24T13:48:34.556 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c1.client.1.keyring && sudo chmod 0644 /etc/ceph/c1.client.1.keyring 2020-05-24T13:48:34.740 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/c1.client.1.keyring 2020-05-24T13:48:34.772 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-05-24T13:48:34.773 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:48:34.773 INFO:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/c1-0 2020-05-24T13:48:34.874 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:34.875 INFO:tasks.ceph:role: c1.osd.0 2020-05-24T13:48:34.876 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:48:34.876 INFO:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-05-24T13:48:35.462 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:48:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:48:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:48:35.463 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:48:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:48:35.463 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:48:35.463 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:48:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:48:35.464 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:48:35.465 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2020-05-24T13:48:35.465 INFO:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c1-0 2020-05-24T13:48:35.659 INFO:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-0 2020-05-24T13:48:35.783 INFO:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/c1-1 2020-05-24T13:48:35.887 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:35.888 INFO:tasks.ceph:role: c1.osd.1 2020-05-24T13:48:35.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:48:35.888 INFO:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-05-24T13:48:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:48:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:48:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:48:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:48:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:48:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:48:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:48:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:48:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:48:36.490 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2020-05-24T13:48:36.490 INFO:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c1-1 2020-05-24T13:48:36.641 INFO:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-1 2020-05-24T13:48:36.753 INFO:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/c1-2 2020-05-24T13:48:36.871 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:36.871 INFO:tasks.ceph:role: c1.osd.2 2020-05-24T13:48:36.872 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:48:36.872 INFO:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-05-24T13:48:37.443 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:48:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:48:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:48:37.444 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:48:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:48:37.444 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:48:37.444 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:48:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:48:37.444 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:48:37.445 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2020-05-24T13:48:37.446 INFO:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c1-2 2020-05-24T13:48:37.616 INFO:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-2 2020-05-24T13:48:37.736 INFO:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:37.918 7f07b70cf9c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-0/keyring: can't open /var/lib/ceph/osd/c1-0/keyring: (2) No such file or directory 2020-05-24T13:48:37.922 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:37.919 7f07b70cf9c0 -1 created new key in keyring /var/lib/ceph/osd/c1-0/keyring 2020-05-24T13:48:37.922 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:37.919 7f07b70cf9c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2020-05-24T13:48:37.922 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:37.919 7f07b70cf9c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2020-05-24T13:48:37.922 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:37.919 7f07b70cf9c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2020-05-24T13:48:37.923 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:37.920 7f07b70cf9c0 -1 bluestore(/var/lib/ceph/osd/c1-0) _read_fsid unparsable uuid 2020-05-24T13:48:39.973 INFO:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-0 2020-05-24T13:48:40.103 INFO:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:40.300 7f95f67939c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-1/keyring: can't open /var/lib/ceph/osd/c1-1/keyring: (2) No such file or directory 2020-05-24T13:48:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:40.301 7f95f67939c0 -1 created new key in keyring /var/lib/ceph/osd/c1-1/keyring 2020-05-24T13:48:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:40.301 7f95f67939c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2020-05-24T13:48:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:40.301 7f95f67939c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2020-05-24T13:48:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:40.301 7f95f67939c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2020-05-24T13:48:40.306 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:40.302 7f95f67939c0 -1 bluestore(/var/lib/ceph/osd/c1-1) _read_fsid unparsable uuid 2020-05-24T13:48:42.360 INFO:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-1 2020-05-24T13:48:42.481 INFO:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:42.679 7ff9887369c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-2/keyring: can't open /var/lib/ceph/osd/c1-2/keyring: (2) No such file or directory 2020-05-24T13:48:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:42.680 7ff9887369c0 -1 created new key in keyring /var/lib/ceph/osd/c1-2/keyring 2020-05-24T13:48:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:42.680 7ff9887369c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2020-05-24T13:48:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:42.680 7ff9887369c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2020-05-24T13:48:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:42.680 7ff9887369c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2020-05-24T13:48:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:48:42.681 7ff9887369c0 -1 bluestore(/var/lib/ceph/osd/c1-2) _read_fsid unparsable uuid 2020-05-24T13:48:44.738 INFO:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-2 2020-05-24T13:48:44.858 INFO:tasks.ceph:Reading keys from all nodes... 2020-05-24T13:48:44.859 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:48:44.959 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.PesEHw4lln 2020-05-24T13:48:44.960 INFO:teuthology.orchestra.run.smithi019:> sudo cp /var/lib/ceph/mgr/c1-x/keyring /tmp/tmp.PesEHw4lln 2020-05-24T13:48:45.066 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.PesEHw4lln 2020-05-24T13:48:45.259 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.PesEHw4lln is 56B 2020-05-24T13:48:45.364 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.PesEHw4lln 2020-05-24T13:48:45.458 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:48:45.537 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.BDE7VlM6LB 2020-05-24T13:48:45.538 INFO:teuthology.orchestra.run.smithi019:> sudo cp /var/lib/ceph/osd/c1-0/keyring /tmp/tmp.BDE7VlM6LB 2020-05-24T13:48:45.636 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.BDE7VlM6LB 2020-05-24T13:48:45.808 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.BDE7VlM6LB is 56B 2020-05-24T13:48:45.881 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.BDE7VlM6LB 2020-05-24T13:48:45.960 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:48:46.046 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.w7T8i2o5Tj 2020-05-24T13:48:46.046 INFO:teuthology.orchestra.run.smithi019:> sudo cp /var/lib/ceph/osd/c1-1/keyring /tmp/tmp.w7T8i2o5Tj 2020-05-24T13:48:46.159 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.w7T8i2o5Tj 2020-05-24T13:48:46.352 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.w7T8i2o5Tj is 56B 2020-05-24T13:48:46.437 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.w7T8i2o5Tj 2020-05-24T13:48:46.517 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:48:46.643 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.hS8pIhwKk9 2020-05-24T13:48:46.644 INFO:teuthology.orchestra.run.smithi019:> sudo cp /var/lib/ceph/osd/c1-2/keyring /tmp/tmp.hS8pIhwKk9 2020-05-24T13:48:46.754 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.hS8pIhwKk9 2020-05-24T13:48:46.944 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.hS8pIhwKk9 is 56B 2020-05-24T13:48:47.036 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.hS8pIhwKk9 2020-05-24T13:48:47.204 DEBUG:teuthology.orchestra.remote:smithi019:/etc/ceph/c1.client.0.keyring is 59B 2020-05-24T13:48:47.352 DEBUG:teuthology.orchestra.remote:smithi019:/etc/ceph/c1.client.1.keyring is 59B 2020-05-24T13:48:47.467 INFO:tasks.ceph:Adding keys to all mons... 2020-05-24T13:48:47.558 INFO:teuthology.orchestra.run.smithi019:> sudo tee -a /etc/ceph/c1.keyring 2020-05-24T13:48:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.x] 2020-05-24T13:48:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAye8petzIQFBAAv5Q4OHYbiy3aIHwcNurxbQ== 2020-05-24T13:48:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2020-05-24T13:48:47.685 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQA1e8peiI7WNhAA1nfWO11nBt8weribgHvz0w== 2020-05-24T13:48:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2020-05-24T13:48:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQA4e8peBZ4BEhAA/lgZ0Se2z3PTFHdquEs3Fw== 2020-05-24T13:48:47.686 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2020-05-24T13:48:47.686 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQA6e8peDdSbKBAA4/E/X/NQSjt4qhkL4oFopA== 2020-05-24T13:48:47.687 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2020-05-24T13:48:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAye8peydpBHxAAlX49rqqyAPc4sWtidcy1qQ== 2020-05-24T13:48:47.687 INFO:teuthology.orchestra.run.smithi019.stdout:[client.1] 2020-05-24T13:48:47.687 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAye8pevocSLBAAc9ofxkeqeHUfAtgIfZxqwA== 2020-05-24T13:48:47.689 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-24T13:48:47.891 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:48:48.088 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:48:48.296 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:48:48.498 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:48:48.703 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:48:48.910 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-05-24T13:48:48.911 INFO:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/c1-a 2020-05-24T13:48:49.013 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c1 --mkfs -i a --monmap /home/ubuntu/cephtest/c1.monmap --keyring /etc/ceph/c1.keyring 2020-05-24T13:48:49.178 INFO:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c1-a 2020-05-24T13:48:49.271 INFO:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:49.342 INFO:tasks.ceph:Starting mon daemons in cluster c1... 2020-05-24T13:48:49.343 DEBUG:teuthology.misc:running c1.mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2020-05-24T13:48:49.344 INFO:tasks.ceph.c1.mon.a:Restarting daemon 2020-05-24T13:48:49.344 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.mon.a.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster c1 -i a 2020-05-24T13:48:49.348 INFO:tasks.ceph.c1.mon.a:Started 2020-05-24T13:48:49.349 INFO:tasks.ceph:Starting mgr daemons in cluster c1... 2020-05-24T13:48:49.349 INFO:tasks.ceph.c1.mgr.x:Restarting daemon 2020-05-24T13:48:49.350 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster c1 -i x 2020-05-24T13:48:49.352 INFO:tasks.ceph.c1.mgr.x:Started 2020-05-24T13:48:49.353 INFO:tasks.ceph:Setting crush tunables to default 2020-05-24T13:48:49.353 INFO:teuthology.orchestra.run.smithi019:> sudo ceph --cluster c1 osd crush tunables default 2020-05-24T13:48:55.088 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:00:05.300 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T13:49:01.154 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2020-05-24T13:49:01.173 INFO:tasks.ceph:Starting osd daemons in cluster c1... 2020-05-24T13:49:01.174 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:49:01.238 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.CIor5uTag3 2020-05-24T13:49:01.239 INFO:teuthology.orchestra.run.smithi019:> sudo cp /var/lib/ceph/osd/c1-0/fsid /tmp/tmp.CIor5uTag3 2020-05-24T13:49:01.318 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.CIor5uTag3 2020-05-24T13:49:01.426 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.CIor5uTag3 is 37B 2020-05-24T13:49:01.476 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.CIor5uTag3 2020-05-24T13:49:01.519 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:49:01.607 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.CKCPv0gKpO 2020-05-24T13:49:01.608 INFO:teuthology.orchestra.run.smithi019:> sudo cp /var/lib/ceph/osd/c1-1/fsid /tmp/tmp.CKCPv0gKpO 2020-05-24T13:49:01.676 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.CKCPv0gKpO 2020-05-24T13:49:01.778 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.CKCPv0gKpO is 37B 2020-05-24T13:49:01.825 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.CKCPv0gKpO 2020-05-24T13:49:01.867 INFO:teuthology.orchestra.run.smithi019:> mktemp 2020-05-24T13:49:01.965 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.RJhnihjyip 2020-05-24T13:49:01.966 INFO:teuthology.orchestra.run.smithi019:> sudo cp /var/lib/ceph/osd/c1-2/fsid /tmp/tmp.RJhnihjyip 2020-05-24T13:49:02.035 INFO:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.RJhnihjyip 2020-05-24T13:49:02.140 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.RJhnihjyip is 37B 2020-05-24T13:49:02.187 INFO:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.RJhnihjyip 2020-05-24T13:49:02.228 INFO:teuthology.orchestra.run.smithi019:> sudo ceph --cluster c1 osd new c1b4a720-617e-4e99-ae23-012ceda35e97 0 2020-05-24T13:49:03.368 INFO:teuthology.orchestra.run.smithi019.stdout:0 2020-05-24T13:49:03.392 INFO:teuthology.orchestra.run.smithi019:> sudo ceph --cluster c1 osd new d37cc6fb-19ba-423a-8ec2-36f70a2e6242 1 2020-05-24T13:49:04.273 INFO:teuthology.orchestra.run.smithi019.stdout:1 2020-05-24T13:49:04.291 INFO:teuthology.orchestra.run.smithi019:> sudo ceph --cluster c1 osd new 91f28dd8-3212-48c4-855b-0a2a4f3f5946 2 2020-05-24T13:49:05.213 INFO:teuthology.orchestra.run.smithi019.stdout:2 2020-05-24T13:49:05.234 DEBUG:teuthology.misc:running c1.osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:49:05.235 INFO:tasks.ceph.c1.osd.0:Restarting daemon 2020-05-24T13:49:05.236 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.osd.0.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c1 -i 0 2020-05-24T13:49:05.240 INFO:tasks.ceph.c1.osd.0:Started 2020-05-24T13:49:05.240 DEBUG:teuthology.misc:running c1.osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:49:05.241 INFO:tasks.ceph.c1.osd.1:Restarting daemon 2020-05-24T13:49:05.241 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.osd.1.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c1 -i 1 2020-05-24T13:49:05.244 INFO:tasks.ceph.c1.osd.1:Started 2020-05-24T13:49:05.244 DEBUG:teuthology.misc:running c1.osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:49:05.245 INFO:tasks.ceph.c1.osd.2:Restarting daemon 2020-05-24T13:49:05.245 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.osd.2.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c1 -i 2 2020-05-24T13:49:05.249 INFO:tasks.ceph.c1.osd.2:Started 2020-05-24T13:49:05.249 INFO:tasks.ceph:Waiting for OSDs to come up 2020-05-24T13:49:05.553 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:06.409 INFO:teuthology.misc.health.smithi019.stdout: 2020-05-24T13:49:06.409 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":5,"fsid":"d75753f9-1100-4c74-b8ad-924f57b3114a","created":"2020-05-24 13:48:57.473791","modified":"2020-05-24 13:49:05.108074","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"c1b4a720-617e-4e99-ae23-012ceda35e97","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"d37cc6fb-19ba-423a-8ec2-36f70a2e6242","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"91f28dd8-3212-48c4-855b-0a2a4f3f5946","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:06.427 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:49:11.643 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:==00:00:00:06.249 10238== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:11.735 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:==00:00:00:06.342 10238== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:11.755 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:==00:00:00:06.384 10236== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:11.816 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:==00:00:00:06.446 10236== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:11.968 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:==00:00:00:06.568 10237== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:12.016 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:==00:00:00:06.624 10237== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:12.313 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:2020-05-24 13:49:12.290 9ae5e40 -1 Falling back to public interface 2020-05-24T13:49:12.432 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:2020-05-24 13:49:12.401 9ae5e40 -1 Falling back to public interface 2020-05-24T13:49:12.559 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:==00:00:00:07.165 10238== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:12.606 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:2020-05-24 13:49:12.578 9ae5e40 -1 Falling back to public interface 2020-05-24T13:49:12.680 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:==00:00:00:07.309 10236== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:12.731 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:12.848 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:==00:00:00:07.456 10237== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:12.974 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:==00:00:00:07.581 10238== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:12.987 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:==00:00:00:07.594 10238== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:13.071 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:==00:00:00:07.701 10236== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:13.080 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:==00:00:00:07.710 10236== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:13.203 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:==00:00:00:07.809 10237== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:13.210 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:==00:00:00:07.818 10237== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:13.652 INFO:teuthology.misc.health.smithi019.stdout: 2020-05-24T13:49:13.652 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":5,"fsid":"d75753f9-1100-4c74-b8ad-924f57b3114a","created":"2020-05-24 13:48:57.473791","modified":"2020-05-24 13:49:05.108074","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"c1b4a720-617e-4e99-ae23-012ceda35e97","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"d37cc6fb-19ba-423a-8ec2-36f70a2e6242","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"91f28dd8-3212-48c4-855b-0a2a4f3f5946","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:13.663 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:49:13.905 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:==00:00:00:08.511 10238== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:14.126 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:==00:00:00:08.734 10237== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:14.167 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:==00:00:00:08.796 10236== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:15.337 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:2020-05-24 13:49:15.332 9ae5e40 -1 osd.0 0 log_to_monitors {default=true} 2020-05-24T13:49:15.517 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:2020-05-24 13:49:15.511 9ae5e40 -1 osd.1 0 log_to_monitors {default=true} 2020-05-24T13:49:15.521 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:2020-05-24 13:49:15.516 9ae5e40 -1 osd.2 0 log_to_monitors {default=true} 2020-05-24T13:49:17.181 INFO:tasks.ceph.c1.osd.0.smithi019.stderr:2020-05-24 13:49:17.158 290bd700 -1 osd.0 0 waiting for initial osdmap 2020-05-24T13:49:18.300 INFO:tasks.ceph.c1.osd.1.smithi019.stderr:2020-05-24 13:49:18.294 28cbc700 -1 osd.1 0 waiting for initial osdmap 2020-05-24T13:49:18.307 INFO:tasks.ceph.c1.osd.2.smithi019.stderr:2020-05-24 13:49:18.294 28cbc700 -1 osd.2 0 waiting for initial osdmap 2020-05-24T13:49:19.966 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:20.890 INFO:teuthology.misc.health.smithi019.stdout: 2020-05-24T13:49:20.892 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":9,"fsid":"d75753f9-1100-4c74-b8ad-924f57b3114a","created":"2020-05-24 13:48:57.473791","modified":"2020-05-24 13:49:19.335870","last_up_change":"2020-05-24 13:49:19.335870","last_in_change":"2020-05-24 13:49:19.335870","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"c1b4a720-617e-4e99-ae23-012ceda35e97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":10238},{"type":"v1","addr":"172.21.15.19:6803","nonce":10238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":10238},{"type":"v1","addr":"172.21.15.19:6805","nonce":10238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":10238},{"type":"v1","addr":"172.21.15.19:6809","nonce":10238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":10238},{"type":"v1","addr":"172.21.15.19:6807","nonce":10238}]},"public_addr":"172.21.15.19:6803/10238","cluster_addr":"172.21.15.19:6805/10238","heartbeat_back_addr":"172.21.15.19:6809/10238","heartbeat_front_addr":"172.21.15.19:6807/10238","state":["exists","up"]},{"osd":1,"uuid":"d37cc6fb-19ba-423a-8ec2-36f70a2e6242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":10236},{"type":"v1","addr":"172.21.15.19:6811","nonce":10236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":10236},{"type":"v1","addr":"172.21.15.19:6813","nonce":10236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":10236},{"type":"v1","addr":"172.21.15.19:6817","nonce":10236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":10236},{"type":"v1","addr":"172.21.15.19:6815","nonce":10236}]},"public_addr":"172.21.15.19:6811/10236","cluster_addr":"172.21.15.19:6813/10236","heartbeat_back_addr":"172.21.15.19:6817/10236","heartbeat_front_addr":"172.21.15.19:6815/10236","state":["exists","up"]},{"osd":2,"uuid":"91f28dd8-3212-48c4-855b-0a2a4f3f5946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":10237},{"type":"v1","addr":"172.21.15.19:6819","nonce":10237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":10237},{"type":"v1","addr":"172.21.15.19:6821","nonce":10237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":10237},{"type":"v1","addr":"172.21.15.19:6825","nonce":10237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":10237},{"type":"v1","addr":"172.21.15.19:6823","nonce":10237}]},"public_addr":"172.21.15.19:6819/10237","cluster_addr":"172.21.15.19:6821/10237","heartbeat_back_addr":"172.21.15.19:6825/10237","heartbeat_front_addr":"172.21.15.19:6823/10237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:20.914 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:49:20.914 INFO:tasks.ceph:Creating RBD pool 2020-05-24T13:49:20.915 INFO:teuthology.orchestra.run.smithi019:> sudo ceph --cluster c1 osd pool create rbd 8 2020-05-24T13:49:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'rbd' created 2020-05-24T13:49:23.065 INFO:teuthology.orchestra.run.smithi019:> sudo ceph --cluster c1 osd pool application enable rbd rbd --yes-i-really-mean-it 2020-05-24T13:49:25.141 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rbd' on pool 'rbd' 2020-05-24T13:49:25.166 INFO:tasks.ceph:Starting mds daemons in cluster c1... 2020-05-24T13:49:25.167 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:25.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:25.984 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"fsid":"d75753f9-1100-4c74-b8ad-924f57b3114a","created":"2020-05-24 13:48:57.473791","modified":"2020-05-24 13:49:24.985307","last_up_change":"2020-05-24 13:49:19.335870","last_in_change":"2020-05-24 13:49:19.335870","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:49:21.848595","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c1b4a720-617e-4e99-ae23-012ceda35e97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":10238},{"type":"v1","addr":"172.21.15.19:6803","nonce":10238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":10238},{"type":"v1","addr":"172.21.15.19:6805","nonce":10238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":10238},{"type":"v1","addr":"172.21.15.19:6809","nonce":10238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":10238},{"type":"v1","addr":"172.21.15.19:6807","nonce":10238}]},"public_addr":"172.21.15.19:6803/10238","cluster_addr":"172.21.15.19:6805/10238","heartbeat_back_addr":"172.21.15.19:6809/10238","heartbeat_front_addr":"172.21.15.19:6807/10238","state":["exists","up"]},{"osd":1,"uuid":"d37cc6fb-19ba-423a-8ec2-36f70a2e6242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":10236},{"type":"v1","addr":"172.21.15.19:6811","nonce":10236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":10236},{"type":"v1","addr":"172.21.15.19:6813","nonce":10236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":10236},{"type":"v1","addr":"172.21.15.19:6817","nonce":10236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":10236},{"type":"v1","addr":"172.21.15.19:6815","nonce":10236}]},"public_addr":"172.21.15.19:6811/10236","cluster_addr":"172.21.15.19:6813/10236","heartbeat_back_addr":"172.21.15.19:6817/10236","heartbeat_front_addr":"172.21.15.19:6815/10236","state":["exists","up"]},{"osd":2,"uuid":"91f28dd8-3212-48c4-855b-0a2a4f3f5946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":10237},{"type":"v1","addr":"172.21.15.19:6819","nonce":10237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":10237},{"type":"v1","addr":"172.21.15.19:6821","nonce":10237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":10237},{"type":"v1","addr":"172.21.15.19:6825","nonce":10237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":10237},{"type":"v1","addr":"172.21.15.19:6823","nonce":10237}]},"public_addr":"172.21.15.19:6819/10237","cluster_addr":"172.21.15.19:6821/10237","heartbeat_back_addr":"172.21.15.19:6825/10237","heartbeat_front_addr":"172.21.15.19:6823/10237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:26.003 INFO:tasks.ceph.ceph_manager.c1:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2020-05-24 13:49:21.848595', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-05-24T13:49:26.004 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd pool get rbd pg_num 2020-05-24T13:49:26.857 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 8 2020-05-24T13:49:26.880 INFO:tasks.ceph:Waiting until c1 daemons up and pgs clean... 2020-05-24T13:49:26.881 INFO:tasks.ceph.ceph_manager.c1:waiting for mgr available 2020-05-24T13:49:26.881 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 mgr dump --format=json 2020-05-24T13:49:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:27.979 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":9345},{"type":"v1","addr":"172.21.15.19:6801","nonce":9345}]},"active_addr":"172.21.15.19:6801/9345","active_change":"2020-05-24 13:49:02.599156","available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout_min":{"name":"inventory_cache_timeout_min","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}} 2020-05-24T13:49:27.981 INFO:tasks.ceph.ceph_manager.c1:mgr available! 2020-05-24T13:49:28.284 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:29.242 INFO:teuthology.misc.health.smithi019.stdout: 2020-05-24T13:49:29.243 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":13,"fsid":"d75753f9-1100-4c74-b8ad-924f57b3114a","created":"2020-05-24 13:48:57.473791","modified":"2020-05-24 13:49:26.142509","last_up_change":"2020-05-24 13:49:19.335870","last_in_change":"2020-05-24 13:49:19.335870","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:49:21.848595","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c1b4a720-617e-4e99-ae23-012ceda35e97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":10238},{"type":"v1","addr":"172.21.15.19:6803","nonce":10238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":10238},{"type":"v1","addr":"172.21.15.19:6805","nonce":10238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":10238},{"type":"v1","addr":"172.21.15.19:6809","nonce":10238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":10238},{"type":"v1","addr":"172.21.15.19:6807","nonce":10238}]},"public_addr":"172.21.15.19:6803/10238","cluster_addr":"172.21.15.19:6805/10238","heartbeat_back_addr":"172.21.15.19:6809/10238","heartbeat_front_addr":"172.21.15.19:6807/10238","state":["exists","up"]},{"osd":1,"uuid":"d37cc6fb-19ba-423a-8ec2-36f70a2e6242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":10236},{"type":"v1","addr":"172.21.15.19:6811","nonce":10236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":10236},{"type":"v1","addr":"172.21.15.19:6813","nonce":10236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":10236},{"type":"v1","addr":"172.21.15.19:6817","nonce":10236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":10236},{"type":"v1","addr":"172.21.15.19:6815","nonce":10236}]},"public_addr":"172.21.15.19:6811/10236","cluster_addr":"172.21.15.19:6813/10236","heartbeat_back_addr":"172.21.15.19:6817/10236","heartbeat_front_addr":"172.21.15.19:6815/10236","state":["exists","up"]},{"osd":2,"uuid":"91f28dd8-3212-48c4-855b-0a2a4f3f5946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":10237},{"type":"v1","addr":"172.21.15.19:6819","nonce":10237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":10237},{"type":"v1","addr":"172.21.15.19:6821","nonce":10237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":10237},{"type":"v1","addr":"172.21.15.19:6825","nonce":10237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":10237},{"type":"v1","addr":"172.21.15.19:6823","nonce":10237}]},"public_addr":"172.21.15.19:6819/10237","cluster_addr":"172.21.15.19:6821/10237","heartbeat_back_addr":"172.21.15.19:6825/10237","heartbeat_front_addr":"172.21.15.19:6823/10237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:29.261 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:49:29.262 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:30.276 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:30.277 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"fsid":"d75753f9-1100-4c74-b8ad-924f57b3114a","created":"2020-05-24 13:48:57.473791","modified":"2020-05-24 13:49:26.142509","last_up_change":"2020-05-24 13:49:19.335870","last_in_change":"2020-05-24 13:49:19.335870","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:49:21.848595","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c1b4a720-617e-4e99-ae23-012ceda35e97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":10238},{"type":"v1","addr":"172.21.15.19:6803","nonce":10238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":10238},{"type":"v1","addr":"172.21.15.19:6805","nonce":10238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":10238},{"type":"v1","addr":"172.21.15.19:6809","nonce":10238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":10238},{"type":"v1","addr":"172.21.15.19:6807","nonce":10238}]},"public_addr":"172.21.15.19:6803/10238","cluster_addr":"172.21.15.19:6805/10238","heartbeat_back_addr":"172.21.15.19:6809/10238","heartbeat_front_addr":"172.21.15.19:6807/10238","state":["exists","up"]},{"osd":1,"uuid":"d37cc6fb-19ba-423a-8ec2-36f70a2e6242","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":10236},{"type":"v1","addr":"172.21.15.19:6811","nonce":10236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":10236},{"type":"v1","addr":"172.21.15.19:6813","nonce":10236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":10236},{"type":"v1","addr":"172.21.15.19:6817","nonce":10236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":10236},{"type":"v1","addr":"172.21.15.19:6815","nonce":10236}]},"public_addr":"172.21.15.19:6811/10236","cluster_addr":"172.21.15.19:6813/10236","heartbeat_back_addr":"172.21.15.19:6817/10236","heartbeat_front_addr":"172.21.15.19:6815/10236","state":["exists","up"]},{"osd":2,"uuid":"91f28dd8-3212-48c4-855b-0a2a4f3f5946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":10237},{"type":"v1","addr":"172.21.15.19:6819","nonce":10237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":10237},{"type":"v1","addr":"172.21.15.19:6821","nonce":10237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":10237},{"type":"v1","addr":"172.21.15.19:6825","nonce":10237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":10237},{"type":"v1","addr":"172.21.15.19:6823","nonce":10237}]},"public_addr":"172.21.15.19:6819/10237","cluster_addr":"172.21.15.19:6821/10237","heartbeat_back_addr":"172.21.15.19:6825/10237","heartbeat_front_addr":"172.21.15.19:6823/10237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:30.300 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 tell osd.0 flush_pg_stats 2020-05-24T13:49:30.928 INFO:teuthology.orchestra.run.smithi019.stdout:34359738372 2020-05-24T13:49:30.946 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 tell osd.1 flush_pg_stats 2020-05-24T13:49:31.522 INFO:teuthology.orchestra.run.smithi019.stdout:38654705669 2020-05-24T13:49:31.544 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 tell osd.2 flush_pg_stats 2020-05-24T13:49:32.164 INFO:teuthology.orchestra.run.smithi019.stdout:38654705668 2020-05-24T13:49:32.180 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd last-stat-seq osd.0 2020-05-24T13:49:33.115 INFO:teuthology.orchestra.run.smithi019.stdout:34359738372 2020-05-24T13:49:33.139 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738372 got 34359738372 for osd.0 2020-05-24T13:49:33.139 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd last-stat-seq osd.1 2020-05-24T13:49:34.004 INFO:teuthology.orchestra.run.smithi019.stdout:38654705669 2020-05-24T13:49:34.028 INFO:tasks.ceph.ceph_manager.c1:need seq 38654705669 got 38654705669 for osd.1 2020-05-24T13:49:34.029 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd last-stat-seq osd.2 2020-05-24T13:49:34.987 INFO:teuthology.orchestra.run.smithi019.stdout:38654705668 2020-05-24T13:49:35.015 INFO:tasks.ceph.ceph_manager.c1:need seq 38654705668 got 38654705668 for osd.2 2020-05-24T13:49:35.015 INFO:tasks.ceph.ceph_manager.c1:waiting for clean 2020-05-24T13:49:35.016 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 pg dump --format=json 2020-05-24T13:49:35.852 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2020-05-24T13:49:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:35.869 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-05-24 13:49:34.914614","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":86703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.808688","last_change":"2020-05-24 13:49:25.689957","last_active":"2020-05-24 13:49:25.808688","last_peered":"2020-05-24 13:49:25.808688","last_clean":"2020-05-24 13:49:25.808688","last_became_active":"2020-05-24 13:49:25.608618","last_became_peered":"2020-05-24 13:49:25.608618","last_unstale":"2020-05-24 13:49:25.808688","last_undegraded":"2020-05-24 13:49:25.808688","last_fullsized":"2020-05-24 13:49:25.808688","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.810940","last_change":"2020-05-24 13:49:25.751954","last_active":"2020-05-24 13:49:25.810940","last_peered":"2020-05-24 13:49:25.810940","last_clean":"2020-05-24 13:49:25.810940","last_became_active":"2020-05-24 13:49:25.705368","last_became_peered":"2020-05-24 13:49:25.705368","last_unstale":"2020-05-24 13:49:25.810940","last_undegraded":"2020-05-24 13:49:25.810940","last_fullsized":"2020-05-24 13:49:25.810940","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.704298","last_change":"2020-05-24 13:49:25.681200","last_active":"2020-05-24 13:49:25.704298","last_peered":"2020-05-24 13:49:25.704298","last_clean":"2020-05-24 13:49:25.704298","last_became_active":"2020-05-24 13:49:25.665198","last_became_peered":"2020-05-24 13:49:25.665198","last_unstale":"2020-05-24 13:49:25.704298","last_undegraded":"2020-05-24 13:49:25.704298","last_fullsized":"2020-05-24 13:49:25.704298","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.809735","last_change":"2020-05-24 13:49:25.742021","last_active":"2020-05-24 13:49:25.809735","last_peered":"2020-05-24 13:49:25.809735","last_clean":"2020-05-24 13:49:25.809735","last_became_active":"2020-05-24 13:49:25.707931","last_became_peered":"2020-05-24 13:49:25.707931","last_unstale":"2020-05-24 13:49:25.809735","last_undegraded":"2020-05-24 13:49:25.809735","last_fullsized":"2020-05-24 13:49:25.809735","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.812627","last_change":"2020-05-24 13:49:25.758104","last_active":"2020-05-24 13:49:25.812627","last_peered":"2020-05-24 13:49:25.812627","last_clean":"2020-05-24 13:49:25.812627","last_became_active":"2020-05-24 13:49:25.615790","last_became_peered":"2020-05-24 13:49:25.615790","last_unstale":"2020-05-24 13:49:25.812627","last_undegraded":"2020-05-24 13:49:25.812627","last_fullsized":"2020-05-24 13:49:25.812627","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.702519","last_change":"2020-05-24 13:49:25.662468","last_active":"2020-05-24 13:49:25.702519","last_peered":"2020-05-24 13:49:25.702519","last_clean":"2020-05-24 13:49:25.702519","last_became_active":"2020-05-24 13:49:25.579494","last_became_peered":"2020-05-24 13:49:25.579494","last_unstale":"2020-05-24 13:49:25.702519","last_undegraded":"2020-05-24 13:49:25.702519","last_fullsized":"2020-05-24 13:49:25.702519","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.512225","last_change":"2020-05-24 13:49:24.750259","last_active":"2020-05-24 13:49:25.512225","last_peered":"2020-05-24 13:49:25.512225","last_clean":"2020-05-24 13:49:25.512225","last_became_active":"2020-05-24 13:49:24.705444","last_became_peered":"2020-05-24 13:49:24.705444","last_unstale":"2020-05-24 13:49:25.512225","last_undegraded":"2020-05-24 13:49:25.512225","last_fullsized":"2020-05-24 13:49:25.512225","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.811620","last_change":"2020-05-24 13:49:25.779847","last_active":"2020-05-24 13:49:25.811620","last_peered":"2020-05-24 13:49:25.811620","last_clean":"2020-05-24 13:49:25.811620","last_became_active":"2020-05-24 13:49:25.580930","last_became_peered":"2020-05-24 13:49:25.580930","last_unstale":"2020-05-24 13:49:25.811620","last_undegraded":"2020-05-24 13:49:25.811620","last_fullsized":"2020-05-24 13:49:25.811620","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:49:35.870 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 pg dump --format=json 2020-05-24T13:49:36.777 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2020-05-24T13:49:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:36.801 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-05-24 13:49:34.914614","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":86703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.808688","last_change":"2020-05-24 13:49:25.689957","last_active":"2020-05-24 13:49:25.808688","last_peered":"2020-05-24 13:49:25.808688","last_clean":"2020-05-24 13:49:25.808688","last_became_active":"2020-05-24 13:49:25.608618","last_became_peered":"2020-05-24 13:49:25.608618","last_unstale":"2020-05-24 13:49:25.808688","last_undegraded":"2020-05-24 13:49:25.808688","last_fullsized":"2020-05-24 13:49:25.808688","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.810940","last_change":"2020-05-24 13:49:25.751954","last_active":"2020-05-24 13:49:25.810940","last_peered":"2020-05-24 13:49:25.810940","last_clean":"2020-05-24 13:49:25.810940","last_became_active":"2020-05-24 13:49:25.705368","last_became_peered":"2020-05-24 13:49:25.705368","last_unstale":"2020-05-24 13:49:25.810940","last_undegraded":"2020-05-24 13:49:25.810940","last_fullsized":"2020-05-24 13:49:25.810940","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.704298","last_change":"2020-05-24 13:49:25.681200","last_active":"2020-05-24 13:49:25.704298","last_peered":"2020-05-24 13:49:25.704298","last_clean":"2020-05-24 13:49:25.704298","last_became_active":"2020-05-24 13:49:25.665198","last_became_peered":"2020-05-24 13:49:25.665198","last_unstale":"2020-05-24 13:49:25.704298","last_undegraded":"2020-05-24 13:49:25.704298","last_fullsized":"2020-05-24 13:49:25.704298","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.809735","last_change":"2020-05-24 13:49:25.742021","last_active":"2020-05-24 13:49:25.809735","last_peered":"2020-05-24 13:49:25.809735","last_clean":"2020-05-24 13:49:25.809735","last_became_active":"2020-05-24 13:49:25.707931","last_became_peered":"2020-05-24 13:49:25.707931","last_unstale":"2020-05-24 13:49:25.809735","last_undegraded":"2020-05-24 13:49:25.809735","last_fullsized":"2020-05-24 13:49:25.809735","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.812627","last_change":"2020-05-24 13:49:25.758104","last_active":"2020-05-24 13:49:25.812627","last_peered":"2020-05-24 13:49:25.812627","last_clean":"2020-05-24 13:49:25.812627","last_became_active":"2020-05-24 13:49:25.615790","last_became_peered":"2020-05-24 13:49:25.615790","last_unstale":"2020-05-24 13:49:25.812627","last_undegraded":"2020-05-24 13:49:25.812627","last_fullsized":"2020-05-24 13:49:25.812627","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.702519","last_change":"2020-05-24 13:49:25.662468","last_active":"2020-05-24 13:49:25.702519","last_peered":"2020-05-24 13:49:25.702519","last_clean":"2020-05-24 13:49:25.702519","last_became_active":"2020-05-24 13:49:25.579494","last_became_peered":"2020-05-24 13:49:25.579494","last_unstale":"2020-05-24 13:49:25.702519","last_undegraded":"2020-05-24 13:49:25.702519","last_fullsized":"2020-05-24 13:49:25.702519","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.512225","last_change":"2020-05-24 13:49:24.750259","last_active":"2020-05-24 13:49:25.512225","last_peered":"2020-05-24 13:49:25.512225","last_clean":"2020-05-24 13:49:25.512225","last_became_active":"2020-05-24 13:49:24.705444","last_became_peered":"2020-05-24 13:49:24.705444","last_unstale":"2020-05-24 13:49:25.512225","last_undegraded":"2020-05-24 13:49:25.512225","last_fullsized":"2020-05-24 13:49:25.512225","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:25.811620","last_change":"2020-05-24 13:49:25.779847","last_active":"2020-05-24 13:49:25.811620","last_peered":"2020-05-24 13:49:25.811620","last_clean":"2020-05-24 13:49:25.811620","last_became_active":"2020-05-24 13:49:25.580930","last_became_peered":"2020-05-24 13:49:25.580930","last_unstale":"2020-05-24 13:49:25.811620","last_undegraded":"2020-05-24 13:49:25.811620","last_fullsized":"2020-05-24 13:49:25.811620","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:49:22.665665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:49:22.665665","last_clean_scrub_stamp":"2020-05-24 13:49:22.665665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:49:36.803 INFO:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 status --format=json-pretty 2020-05-24T13:49:37.844 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:37.844 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2020-05-24T13:49:37.844 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "d75753f9-1100-4c74-b8ad-924f57b3114a", 2020-05-24T13:49:37.845 INFO:teuthology.orchestra.run.smithi019.stdout: "health": { 2020-05-24T13:49:37.845 INFO:teuthology.orchestra.run.smithi019.stdout: "checks": {}, 2020-05-24T13:49:37.845 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "HEALTH_OK" 2020-05-24T13:49:37.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.846 INFO:teuthology.orchestra.run.smithi019.stdout: "election_epoch": 3, 2020-05-24T13:49:37.846 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum": [ 2020-05-24T13:49:37.846 INFO:teuthology.orchestra.run.smithi019.stdout: 0 2020-05-24T13:49:37.846 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:37.846 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_names": [ 2020-05-24T13:49:37.846 INFO:teuthology.orchestra.run.smithi019.stdout: "a" 2020-05-24T13:49:37.847 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:37.847 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_age": 40, 2020-05-24T13:49:37.847 INFO:teuthology.orchestra.run.smithi019.stdout: "monmap": { 2020-05-24T13:49:37.847 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 1, 2020-05-24T13:49:37.847 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "d75753f9-1100-4c74-b8ad-924f57b3114a", 2020-05-24T13:49:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2020-05-24 13:48:32.842342", 2020-05-24T13:49:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2020-05-24 13:48:32.842342", 2020-05-24T13:49:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release": 14, 2020-05-24T13:49:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release_name": "nautilus", 2020-05-24T13:49:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: "features": { 2020-05-24T13:49:37.849 INFO:teuthology.orchestra.run.smithi019.stdout: "persistent": [ 2020-05-24T13:49:37.849 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2020-05-24T13:49:37.849 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2020-05-24T13:49:37.849 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2020-05-24T13:49:37.849 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2020-05-24T13:49:37.849 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus" 2020-05-24T13:49:37.850 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:37.850 INFO:teuthology.orchestra.run.smithi019.stdout: "optional": [] 2020-05-24T13:49:37.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.850 INFO:teuthology.orchestra.run.smithi019.stdout: "mons": [ 2020-05-24T13:49:37.850 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.851 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 0, 2020-05-24T13:49:37.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "a", 2020-05-24T13:49:37.851 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2020-05-24T13:49:37.851 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2020-05-24T13:49:37.851 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2020-05-24T13:49:37.852 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:3300", 2020-05-24T13:49:37.852 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2020-05-24T13:49:37.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2020-05-24T13:49:37.853 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789", 2020-05-24T13:49:37.853 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2020-05-24T13:49:37.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.853 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2020-05-24T13:49:37.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.854 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789/0", 2020-05-24T13:49:37.854 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6789/0" 2020-05-24T13:49:37.854 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.854 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2020-05-24T13:49:37.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.855 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap": { 2020-05-24T13:49:37.855 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap": { 2020-05-24T13:49:37.855 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 13, 2020-05-24T13:49:37.855 INFO:teuthology.orchestra.run.smithi019.stdout: "num_osds": 3, 2020-05-24T13:49:37.855 INFO:teuthology.orchestra.run.smithi019.stdout: "num_up_osds": 3, 2020-05-24T13:49:37.855 INFO:teuthology.orchestra.run.smithi019.stdout: "num_in_osds": 3, 2020-05-24T13:49:37.856 INFO:teuthology.orchestra.run.smithi019.stdout: "num_remapped_pgs": 0 2020-05-24T13:49:37.856 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.856 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap": { 2020-05-24T13:49:37.856 INFO:teuthology.orchestra.run.smithi019.stdout: "pgs_by_state": [ 2020-05-24T13:49:37.857 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.857 INFO:teuthology.orchestra.run.smithi019.stdout: "state_name": "active+clean", 2020-05-24T13:49:37.857 INFO:teuthology.orchestra.run.smithi019.stdout: "count": 8 2020-05-24T13:49:37.857 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.857 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:37.858 INFO:teuthology.orchestra.run.smithi019.stdout: "num_pgs": 8, 2020-05-24T13:49:37.858 INFO:teuthology.orchestra.run.smithi019.stdout: "num_pools": 1, 2020-05-24T13:49:37.859 INFO:teuthology.orchestra.run.smithi019.stdout: "num_objects": 0, 2020-05-24T13:49:37.859 INFO:teuthology.orchestra.run.smithi019.stdout: "data_bytes": 0, 2020-05-24T13:49:37.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bytes_used": 3221594112, 2020-05-24T13:49:37.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bytes_avail": 318900953088, 2020-05-24T13:49:37.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bytes_total": 322122547200 2020-05-24T13:49:37.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.860 INFO:teuthology.orchestra.run.smithi019.stdout: "fsmap": { 2020-05-24T13:49:37.860 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 1, 2020-05-24T13:49:37.860 INFO:teuthology.orchestra.run.smithi019.stdout: "by_rank": [], 2020-05-24T13:49:37.860 INFO:teuthology.orchestra.run.smithi019.stdout: "up:standby": 0 2020-05-24T13:49:37.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.861 INFO:teuthology.orchestra.run.smithi019.stdout: "mgrmap": { 2020-05-24T13:49:37.861 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 3, 2020-05-24T13:49:37.861 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 4102, 2020-05-24T13:49:37.861 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2020-05-24T13:49:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2020-05-24T13:49:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2020-05-24T13:49:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2020-05-24T13:49:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2020-05-24T13:49:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 9345 2020-05-24T13:49:37.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.863 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2020-05-24T13:49:37.863 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2020-05-24T13:49:37.863 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 9345 2020-05-24T13:49:37.864 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.864 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2020-05-24T13:49:37.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.864 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/9345", 2020-05-24T13:49:37.864 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2020-05-24 13:49:02.599156", 2020-05-24T13:49:37.865 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2020-05-24T13:49:37.865 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [], 2020-05-24T13:49:37.865 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2020-05-24T13:49:37.865 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2020-05-24T13:49:37.865 INFO:teuthology.orchestra.run.smithi019.stdout: "restful" 2020-05-24T13:49:37.865 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:37.866 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2020-05-24T13:49:37.866 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2020-05-24T13:49:37.866 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:37.866 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:37.868 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:37.868 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2020-05-24T13:49:37.868 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2020-05-24T13:49:37.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2020-05-24T13:49:37.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2020-05-24T13:49:37.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.871 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2020-05-24T13:49:37.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2020-05-24T13:49:37.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2020-05-24T13:49:37.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.874 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2020-05-24T13:49:37.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2020-05-24T13:49:37.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2020-05-24T13:49:37.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2020-05-24T13:49:37.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.878 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2020-05-24T13:49:37.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2020-05-24T13:49:37.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2020-05-24T13:49:37.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.881 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2020-05-24T13:49:37.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2020-05-24T13:49:37.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.882 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2020-05-24T13:49:37.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.883 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2020-05-24T13:49:37.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2020-05-24T13:49:37.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:37.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2020-05-24T13:49:37.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2020-05-24T13:49:37.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.892 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2020-05-24T13:49:37.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2020-05-24T13:49:37.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2020-05-24T13:49:37.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.895 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2020-05-24T13:49:37.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2020-05-24T13:49:37.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2020-05-24T13:49:37.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.898 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2020-05-24T13:49:37.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2020-05-24T13:49:37.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2020-05-24T13:49:37.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.900 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ansible", 2020-05-24T13:49:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2020-05-24T13:49:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2020-05-24T13:49:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "server_url": { 2020-05-24T13:49:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_url", 2020-05-24T13:49:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2020-05-24T13:49:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2020-05-24T13:49:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_server": { 2020-05-24T13:49:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_server", 2020-05-24T13:49:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2020-05-24T13:49:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2020-05-24T13:49:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2020-05-24T13:49:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2020-05-24T13:49:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2020-05-24T13:49:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2020-05-24T13:49:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2020-05-24T13:49:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2020-05-24T13:49:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:49:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2020-05-24T13:49:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2020-05-24T13:49:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2020-05-24T13:49:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2020-05-24T13:49:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2020-05-24T13:49:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "7", 2020-05-24T13:49:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2020-05-24T13:49:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:49:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2020-05-24T13:49:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2020-05-24T13:49:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2020-05-24T13:49:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2020-05-24T13:49:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2020-05-24T13:49:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2020-05-24T13:49:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2020-05-24T13:49:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2020-05-24T13:49:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2020-05-24T13:49:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2020-05-24T13:49:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2020-05-24T13:49:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2020-05-24T13:49:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2020-05-24T13:49:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2020-05-24T13:49:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2020-05-24T13:49:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2020-05-24T13:49:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2020-05-24T13:49:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2020-05-24T13:49:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2020-05-24T13:49:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2020-05-24T13:49:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2020-05-24T13:49:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2020-05-24T13:49:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2400", 2020-05-24T13:49:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2020-05-24T13:49:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:49:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2020-05-24T13:49:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2020-05-24T13:49:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2020-05-24T13:49:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2020-05-24T13:49:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2020-05-24T13:49:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "7", 2020-05-24T13:49:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2020-05-24T13:49:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:49:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2020-05-24T13:49:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2020-05-24T13:49:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2020-05-24T13:49:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2020-05-24T13:49:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2020-05-24T13:49:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2020-05-24T13:49:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2020-05-24T13:49:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "none", 2020-05-24T13:49:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2020-05-24T13:49:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2020-05-24T13:49:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2020-05-24T13:49:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap" 2020-05-24T13:49:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2020-05-24T13:49:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2020-05-24T13:49:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2020-05-24T13:49:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2020-05-24T13:49:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2020-05-24T13:49:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2020-05-24T13:49:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2020-05-24T13:49:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2020-05-24T13:49:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2020-05-24T13:49:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2020-05-24T13:49:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2020-05-24T13:49:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2020-05-24T13:49:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2020-05-24T13:49:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2020-05-24T13:49:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_iterations": { 2020-05-24T13:49:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_iterations", 2020-05-24T13:49:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2020-05-24T13:49:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2020-05-24T13:49:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimization iterations", 2020-05-24T13:49:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2020-05-24T13:49:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2020-05-24T13:49:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2020-05-24T13:49:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2020-05-24T13:49:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2020-05-24T13:49:37.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.967 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2020-05-24T13:49:37.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2020-05-24T13:49:37.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:37.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2020-05-24T13:49:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2020-05-24T13:49:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2020-05-24T13:49:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2020-05-24T13:49:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2020-05-24T13:49:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2020-05-24T13:49:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2020-05-24T13:49:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2020-05-24T13:49:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2020-05-24T13:49:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2020-05-24T13:49:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2020-05-24T13:49:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_cephfs": { 2020-05-24T13:49:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_cephfs", 2020-05-24T13:49:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_iscsi": { 2020-05-24T13:49:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_iscsi", 2020-05-24T13:49:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_mirroring": { 2020-05-24T13:49:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_mirroring", 2020-05-24T13:49:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_rbd": { 2020-05-24T13:49:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_rbd", 2020-05-24T13:49:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_rgw": { 2020-05-24T13:49:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_rgw", 2020-05-24T13:49:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2020-05-24T13:49:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2020-05-24T13:49:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2020-05-24T13:49:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2020-05-24T13:49:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2020-05-24T13:49:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2020-05-24T13:49:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2020-05-24T13:49:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2020-05-24T13:49:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2020-05-24T13:49:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2020-05-24T13:49:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2020-05-24T13:49:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2020-05-24T13:49:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2020-05-24T13:49:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2020-05-24T13:49:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2020-05-24T13:49:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2020-05-24T13:49:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.020 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2020-05-24T13:49:38.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2020-05-24T13:49:38.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.023 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2020-05-24T13:49:38.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2020-05-24T13:49:38.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2020-05-24T13:49:38.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.026 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2020-05-24T13:49:38.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2020-05-24T13:49:38.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.029 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2020-05-24T13:49:38.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2020-05-24T13:49:38.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2020-05-24T13:49:38.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.033 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_HOST": { 2020-05-24T13:49:38.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_HOST", 2020-05-24T13:49:38.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.035 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_PORT": { 2020-05-24T13:49:38.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_PORT", 2020-05-24T13:49:38.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "80", 2020-05-24T13:49:38.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.038 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SCHEME": { 2020-05-24T13:49:38.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SCHEME", 2020-05-24T13:49:38.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "http", 2020-05-24T13:49:38.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.042 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2020-05-24T13:49:38.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2020-05-24T13:49:38.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.045 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2020-05-24T13:49:38.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2020-05-24T13:49:38.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.047 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_USER_ID": { 2020-05-24T13:49:38.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_USER_ID", 2020-05-24T13:49:38.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.051 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2020-05-24T13:49:38.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2020-05-24T13:49:38.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.054 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2020-05-24T13:49:38.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2020-05-24T13:49:38.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:38.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2020-05-24T13:49:38.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.057 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2020-05-24T13:49:38.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2020-05-24T13:49:38.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2020-05-24T13:49:38.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.060 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2020-05-24T13:49:38.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2020-05-24T13:49:38.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.063 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2020-05-24T13:49:38.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2020-05-24T13:49:38.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.066 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2020-05-24T13:49:38.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2020-05-24T13:49:38.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2020-05-24T13:49:38.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.069 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2020-05-24T13:49:38.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2020-05-24T13:49:38.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2020-05-24T13:49:38.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.072 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2020-05-24T13:49:38.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2020-05-24T13:49:38.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.075 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2020-05-24T13:49:38.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2020-05-24T13:49:38.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2020-05-24T13:49:38.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.078 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2020-05-24T13:49:38.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2020-05-24T13:49:38.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2020-05-24T13:49:38.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2020-05-24T13:49:38.080 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2020-05-24T13:49:38.080 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2020-05-24T13:49:38.080 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:38.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.082 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2020-05-24T13:49:38.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2020-05-24T13:49:38.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2020-05-24T13:49:38.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2020-05-24T13:49:38.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2020-05-24T13:49:38.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.085 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2020-05-24T13:49:38.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2020-05-24T13:49:38.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.088 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2020-05-24T13:49:38.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2020-05-24T13:49:38.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.091 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.091 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.092 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "deepsea", 2020-05-24T13:49:38.092 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.092 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.093 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.093 INFO:teuthology.orchestra.run.smithi019.stdout: "salt_api_eauth": { 2020-05-24T13:49:38.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "salt_api_eauth", 2020-05-24T13:49:38.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "sharedsecret", 2020-05-24T13:49:38.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.096 INFO:teuthology.orchestra.run.smithi019.stdout: "salt_api_password": { 2020-05-24T13:49:38.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "salt_api_password", 2020-05-24T13:49:38.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.099 INFO:teuthology.orchestra.run.smithi019.stdout: "salt_api_url": { 2020-05-24T13:49:38.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "salt_api_url", 2020-05-24T13:49:38.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.102 INFO:teuthology.orchestra.run.smithi019.stdout: "salt_api_username": { 2020-05-24T13:49:38.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "salt_api_username", 2020-05-24T13:49:38.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.105 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.105 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2020-05-24T13:49:38.106 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.106 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.107 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2020-05-24T13:49:38.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2020-05-24T13:49:38.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:38.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2020-05-24T13:49:38.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.110 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2020-05-24T13:49:38.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2020-05-24T13:49:38.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2020-05-24T13:49:38.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2020-05-24T13:49:38.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.114 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2020-05-24T13:49:38.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2020-05-24T13:49:38.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2020-05-24T13:49:38.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2020-05-24T13:49:38.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.117 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2020-05-24T13:49:38.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2020-05-24T13:49:38.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2020-05-24T13:49:38.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2020-05-24T13:49:38.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.120 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2020-05-24T13:49:38.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2020-05-24T13:49:38.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2020-05-24T13:49:38.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2020-05-24T13:49:38.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2020-05-24T13:49:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2020-05-24T13:49:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2020-05-24T13:49:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2020-05-24T13:49:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2020-05-24T13:49:38.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2020-05-24T13:49:38.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2020-05-24T13:49:38.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.129 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2020-05-24T13:49:38.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2020-05-24T13:49:38.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2020-05-24T13:49:38.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2020-05-24T13:49:38.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_cloud", 2020-05-24T13:49:38.133 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.133 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.134 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.134 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_cert_context": { 2020-05-24T13:49:38.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_cert_context", 2020-05-24T13:49:38.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.137 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_default_authority": { 2020-05-24T13:49:38.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_default_authority", 2020-05-24T13:49:38.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "localhost", 2020-05-24T13:49:38.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.140 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_password": { 2020-05-24T13:49:38.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_password", 2020-05-24T13:49:38.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.143 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_port": { 2020-05-24T13:49:38.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_port", 2020-05-24T13:49:38.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31400", 2020-05-24T13:49:38.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.146 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_retrieve_prediction_interval": { 2020-05-24T13:49:38.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_retrieve_prediction_interval", 2020-05-24T13:49:38.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "43200", 2020-05-24T13:49:38.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.149 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_server": { 2020-05-24T13:49:38.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_server", 2020-05-24T13:49:38.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.152 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_ssl_target_name_override": { 2020-05-24T13:49:38.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_ssl_target_name_override", 2020-05-24T13:49:38.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "localhost", 2020-05-24T13:49:38.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.155 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_upload_metrics_interval": { 2020-05-24T13:49:38.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_upload_metrics_interval", 2020-05-24T13:49:38.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2020-05-24T13:49:38.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.158 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_upload_smart_interval": { 2020-05-24T13:49:38.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_upload_smart_interval", 2020-05-24T13:49:38.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "43200", 2020-05-24T13:49:38.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.163 INFO:teuthology.orchestra.run.smithi019.stdout: "diskprediction_user": { 2020-05-24T13:49:38.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_user", 2020-05-24T13:49:38.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.167 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2020-05-24T13:49:38.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2020-05-24T13:49:38.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2020-05-24T13:49:38.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2020-05-24T13:49:38.171 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.171 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.171 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.171 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2020-05-24T13:49:38.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2020-05-24T13:49:38.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2020-05-24T13:49:38.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.174 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2020-05-24T13:49:38.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2020-05-24T13:49:38.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2020-05-24T13:49:38.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.177 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.178 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2020-05-24T13:49:38.179 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2020-05-24T13:49:38.179 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2020-05-24T13:49:38.179 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.179 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2020-05-24T13:49:38.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2020-05-24T13:49:38.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2020-05-24T13:49:38.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.182 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2020-05-24T13:49:38.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2020-05-24T13:49:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2020-05-24T13:49:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.185 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2020-05-24T13:49:38.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2020-05-24T13:49:38.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.188 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2020-05-24T13:49:38.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2020-05-24T13:49:38.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2020-05-24T13:49:38.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.191 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2020-05-24T13:49:38.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2020-05-24T13:49:38.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.194 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2020-05-24T13:49:38.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2020-05-24T13:49:38.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2020-05-24T13:49:38.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.197 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2020-05-24T13:49:38.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2020-05-24T13:49:38.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2020-05-24T13:49:38.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.200 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2020-05-24T13:49:38.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2020-05-24T13:49:38.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2020-05-24T13:49:38.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.203 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.203 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.204 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2020-05-24T13:49:38.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2020-05-24T13:49:38.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.207 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2020-05-24T13:49:38.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2020-05-24T13:49:38.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2020-05-24T13:49:38.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2020-05-24T13:49:38.211 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.211 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.211 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": {} 2020-05-24T13:49:38.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2020-05-24T13:49:38.212 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.212 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.212 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": {} 2020-05-24T13:49:38.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2020-05-24T13:49:38.213 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2020-05-24T13:49:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2020-05-24T13:49:38.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2020-05-24T13:49:38.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2020-05-24T13:49:38.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2020-05-24T13:49:38.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2020-05-24T13:49:38.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2020-05-24T13:49:38.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.220 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2020-05-24T13:49:38.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2020-05-24T13:49:38.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2020-05-24T13:49:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2020-05-24T13:49:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.223 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2020-05-24T13:49:38.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2020-05-24T13:49:38.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2020-05-24T13:49:38.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2020-05-24T13:49:38.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.226 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2020-05-24T13:49:38.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2020-05-24T13:49:38.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2020-05-24T13:49:38.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.229 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2020-05-24T13:49:38.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2020-05-24T13:49:38.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2020-05-24T13:49:38.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2020-05-24T13:49:38.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.232 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.232 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator_cli", 2020-05-24T13:49:38.233 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.233 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.233 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.234 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2020-05-24T13:49:38.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2020-05-24T13:49:38.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.237 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.237 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2020-05-24T13:49:38.238 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.238 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.238 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.239 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2020-05-24T13:49:38.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2020-05-24T13:49:38.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2020-05-24T13:49:38.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.241 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.242 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2020-05-24T13:49:38.242 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.243 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.243 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2020-05-24T13:49:38.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2020-05-24T13:49:38.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2020-05-24T13:49:38.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2020-05-24T13:49:38.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.247 INFO:teuthology.orchestra.run.smithi019.stdout: "persist_interval": { 2020-05-24T13:49:38.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "persist_interval", 2020-05-24T13:49:38.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2020-05-24T13:49:38.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2020-05-24T13:49:38.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to persist completed events", 2020-05-24T13:49:38.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2020-05-24T13:49:38.251 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.251 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.251 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.252 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2020-05-24T13:49:38.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2020-05-24T13:49:38.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.255 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2020-05-24T13:49:38.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2020-05-24T13:49:38.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.258 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2020-05-24T13:49:38.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2020-05-24T13:49:38.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.261 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2020-05-24T13:49:38.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2020-05-24T13:49:38.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.264 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2020-05-24T13:49:38.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2020-05-24T13:49:38.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.266 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2020-05-24T13:49:38.267 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.268 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.268 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": {} 2020-05-24T13:49:38.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2020-05-24T13:49:38.269 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.270 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.270 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2020-05-24T13:49:38.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2020-05-24T13:49:38.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.273 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2020-05-24T13:49:38.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2020-05-24T13:49:38.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.276 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2020-05-24T13:49:38.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2020-05-24T13:49:38.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2020-05-24T13:49:38.280 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.280 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.280 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": {} 2020-05-24T13:49:38.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2020-05-24T13:49:38.281 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.282 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.282 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.282 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2020-05-24T13:49:38.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2020-05-24T13:49:38.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.285 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2020-05-24T13:49:38.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2020-05-24T13:49:38.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2020-05-24T13:49:38.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.288 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2020-05-24T13:49:38.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2020-05-24T13:49:38.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.292 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2020-05-24T13:49:38.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2020-05-24T13:49:38.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.295 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2020-05-24T13:49:38.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2020-05-24T13:49:38.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2020-05-24T13:49:38.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2020-05-24T13:49:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2020-05-24T13:49:38.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.301 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2020-05-24T13:49:38.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2020-05-24T13:49:38.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.305 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2020-05-24T13:49:38.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2020-05-24T13:49:38.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.308 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2020-05-24T13:49:38.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2020-05-24T13:49:38.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.311 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2020-05-24T13:49:38.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2020-05-24T13:49:38.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.314 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2020-05-24T13:49:38.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2020-05-24T13:49:38.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh", 2020-05-24T13:49:38.318 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.318 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.318 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.318 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_cache_timeout_min": { 2020-05-24T13:49:38.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_cache_timeout_min", 2020-05-24T13:49:38.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.322 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2020-05-24T13:49:38.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2020-05-24T13:49:38.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2020-05-24T13:49:38.326 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.326 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.326 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": {} 2020-05-24T13:49:38.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2020-05-24T13:49:38.327 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.327 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.327 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.328 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2020-05-24T13:49:38.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2020-05-24T13:49:38.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2020-05-24T13:49:38.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.331 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2020-05-24T13:49:38.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2020-05-24T13:49:38.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2020-05-24T13:49:38.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.334 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2020-05-24T13:49:38.334 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.335 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.335 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.335 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2020-05-24T13:49:38.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2020-05-24T13:49:38.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2020-05-24T13:49:38.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.338 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2020-05-24T13:49:38.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2020-05-24T13:49:38.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.341 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2020-05-24T13:49:38.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2020-05-24T13:49:38.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2020-05-24T13:49:38.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.345 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2020-05-24T13:49:38.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2020-05-24T13:49:38.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:38.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.348 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2020-05-24T13:49:38.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2020-05-24T13:49:38.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.351 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2020-05-24T13:49:38.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2020-05-24T13:49:38.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.354 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2020-05-24T13:49:38.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2020-05-24T13:49:38.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2020-05-24T13:49:38.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2020-05-24T13:49:38.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2020-05-24T13:49:38.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:38.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.360 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2020-05-24T13:49:38.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2020-05-24T13:49:38.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2020-05-24T13:49:38.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2020-05-24T13:49:38.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.363 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2020-05-24T13:49:38.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2020-05-24T13:49:38.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2020-05-24T13:49:38.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.366 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2020-05-24T13:49:38.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2020-05-24T13:49:38.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2020-05-24T13:49:38.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2020-05-24T13:49:38.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.369 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2020-05-24T13:49:38.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2020-05-24T13:49:38.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.373 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2020-05-24T13:49:38.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2020-05-24T13:49:38.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.376 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2020-05-24T13:49:38.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2020-05-24T13:49:38.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2020-05-24T13:49:38.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2020-05-24T13:49:38.380 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.380 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.380 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": {} 2020-05-24T13:49:38.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2020-05-24T13:49:38.381 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.381 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.382 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": {} 2020-05-24T13:49:38.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2020-05-24T13:49:38.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2020-05-24T13:49:38.382 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2020-05-24T13:49:38.383 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2020-05-24T13:49:38.383 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2020-05-24T13:49:38.383 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2020-05-24T13:49:38.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2020-05-24T13:49:38.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2020-05-24T13:49:38.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.386 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2020-05-24T13:49:38.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2020-05-24T13:49:38.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2020-05-24T13:49:38.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2020-05-24T13:49:38.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.389 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2020-05-24T13:49:38.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2020-05-24T13:49:38.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "None", 2020-05-24T13:49:38.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.392 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2020-05-24T13:49:38.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2020-05-24T13:49:38.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2020-05-24T13:49:38.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2020-05-24T13:49:38.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.396 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2020-05-24T13:49:38.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2020-05-24T13:49:38.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2020-05-24T13:49:38.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2020-05-24T13:49:38.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2020-05-24T13:49:38.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2020-05-24T13:49:38.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2020-05-24T13:49:38.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2020-05-24T13:49:38.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2020-05-24T13:49:38.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2020-05-24T13:49:38.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2020-05-24T13:49:38.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2020-05-24T13:49:38.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2020-05-24T13:49:38.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.400 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.400 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2020-05-24T13:49:38.400 INFO:teuthology.orchestra.run.smithi019.stdout: "services": {}, 2020-05-24T13:49:38.400 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2020-05-24T13:49:38.400 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus": [ 2020-05-24T13:49:38.401 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2020-05-24T13:49:38.401 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2020-05-24T13:49:38.401 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2020-05-24T13:49:38.401 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator_cli", 2020-05-24T13:49:38.401 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2020-05-24T13:49:38.402 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2020-05-24T13:49:38.402 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2020-05-24T13:49:38.402 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2020-05-24T13:49:38.402 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2020-05-24T13:49:38.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2020-05-24T13:49:38.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.403 INFO:teuthology.orchestra.run.smithi019.stdout: "servicemap": { 2020-05-24T13:49:38.403 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 1, 2020-05-24T13:49:38.403 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "0.000000", 2020-05-24T13:49:38.404 INFO:teuthology.orchestra.run.smithi019.stdout: "services": {} 2020-05-24T13:49:38.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2020-05-24T13:49:38.404 INFO:teuthology.orchestra.run.smithi019.stdout: "progress_events": {} 2020-05-24T13:49:38.404 INFO:teuthology.orchestra.run.smithi019.stdout:} 2020-05-24T13:49:38.407 INFO:tasks.ceph.ceph_manager.c1:{'fsid': 'd75753f9-1100-4c74-b8ad-924f57b3114a', 'health': {'checks': {}, 'status': 'HEALTH_OK'}, 'election_epoch': 3, 'quorum': [0], 'quorum_names': ['a'], 'quorum_age': 40, 'monmap': {'epoch': 1, 'fsid': 'd75753f9-1100-4c74-b8ad-924f57b3114a', 'modified': '2020-05-24 13:48:32.842342', 'created': '2020-05-24 13:48:32.842342', 'min_mon_release': 14, 'min_mon_release_name': 'nautilus', 'features': {'persistent': ['kraken', 'luminous', 'mimic', 'osdmap-prune', 'nautilus'], 'optional': []}, 'mons': [{'rank': 0, 'name': 'a', 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:3300', 'nonce': 0}, {'type': 'v1', 'addr': '172.21.15.19:6789', 'nonce': 0}]}, 'addr': '172.21.15.19:6789/0', 'public_addr': '172.21.15.19:6789/0'}]}, 'osdmap': {'osdmap': {'epoch': 13, 'num_osds': 3, 'num_up_osds': 3, 'num_in_osds': 3, 'num_remapped_pgs': 0}}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 8}], 'num_pgs': 8, 'num_pools': 1, 'num_objects': 0, 'data_bytes': 0, 'bytes_used': 3221594112, 'bytes_avail': 318900953088, 'bytes_total': 322122547200}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'epoch': 3, 'active_gid': 4102, 'active_name': 'x', 'active_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6800', 'nonce': 9345}, {'type': 'v1', 'addr': '172.21.15.19:6801', 'nonce': 9345}]}, 'active_addr': '172.21.15.19:6801/9345', 'active_change': '2020-05-24 13:49:02.599156', 'available': True, 'standbys': [], 'modules': ['iostat', 'restful'], 'available_modules': [{'name': 'alerts', 'can_run': True, 'error_string': '', 'module_options': {'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'How frequently to reexamine health status', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_destination': {'name': 'smtp_destination', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email address to send alerts to', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_from_name': {'name': 'smtp_from_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'Ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email From: name', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_host': {'name': 'smtp_host', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_password': {'name': 'smtp_password', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Password to authenticate with', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_port': {'name': 'smtp_port', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '465', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP port', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_sender': {'name': 'smtp_sender', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP envelope sender', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_ssl': {'name': 'smtp_ssl', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Use SSL to connect to SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_user': {'name': 'smtp_user', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'User to authenticate as', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ansible', 'can_run': True, 'error_string': '', 'module_options': {'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_url': {'name': 'server_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_server': {'name': 'verify_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'balancer', 'can_run': True, 'error_string': '', 'module_options': {'active': {'name': 'active', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically balance PGs across cluster', 'long_desc': '', 'tags': [], 'see_also': []}, 'begin_time': {'name': 'begin_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '0000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'beginning time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'begin_weekday': {'name': 'begin_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to this day of the week or later', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'crush_compat_max_iterations': {'name': 'crush_compat_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '25', 'min': '1', 'max': '250', 'enum_allowed': [], 'desc': 'maximum number of iterations to attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'crush_compat_metrics': {'name': 'crush_compat_metrics', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'pgs,objects,bytes', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'metrics with which to calculate OSD utilization', 'long_desc': 'Value is a list of one or more of "pgs", "objects", or "bytes", and indicates which metrics to use to balance utilization.', 'tags': [], 'see_also': []}, 'crush_compat_step': {'name': 'crush_compat_step', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0.5', 'min': '0.001', 'max': '0.999', 'enum_allowed': [], 'desc': 'aggressiveness of optimization', 'long_desc': '.99 is very aggressive, .01 is less aggressive', 'tags': [], 'see_also': []}, 'end_time': {'name': 'end_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '2400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'ending time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'end_weekday': {'name': 'end_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '7', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to days of the week earlier than this', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'min_score': {'name': 'min_score', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'minimum score, below which no optimization is attempted', 'long_desc': '', 'tags': [], 'see_also': []}, 'mode': {'name': 'mode', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'none', 'min': '', 'max': '', 'enum_allowed': ['crush-compat', 'none', 'upmap'], 'desc': 'Balancer mode', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_ids': {'name': 'pool_ids', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'pools which the automatic balancing will be limited to', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'upmap_max_deviation': {'name': 'upmap_max_deviation', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '1', 'max': '', 'enum_allowed': [], 'desc': 'deviation below which no optimization is attempted', 'long_desc': 'If the number of PGs are within this count then no optimization is attempted', 'tags': [], 'see_also': []}, 'upmap_max_iterations': {'name': 'upmap_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '10', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'maximum upmap optimization iterations', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'crash', 'can_run': True, 'error_string': '', 'module_options': {'retain_interval': {'name': 'retain_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '31536000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain crashes before pruning them', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_recent_interval': {'name': 'warn_recent_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '1209600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'time interval in which to warn about recent crashes', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'dashboard', 'can_run': True, 'error_string': '', 'module_options': {'ALERTMANAGER_API_HOST': {'name': 'ALERTMANAGER_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_ENABLED': {'name': 'AUDIT_API_ENABLED', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_LOG_PAYLOAD': {'name': 'AUDIT_API_LOG_PAYLOAD', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ENABLE_BROWSABLE_API': {'name': 'ENABLE_BROWSABLE_API', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_cephfs': {'name': 'FEATURE_TOGGLE_cephfs', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_iscsi': {'name': 'FEATURE_TOGGLE_iscsi', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_mirroring': {'name': 'FEATURE_TOGGLE_mirroring', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rbd': {'name': 'FEATURE_TOGGLE_rbd', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rgw': {'name': 'FEATURE_TOGGLE_rgw', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE': {'name': 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_PASSWORD': {'name': 'GRAFANA_API_PASSWORD', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_SSL_VERIFY': {'name': 'GRAFANA_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_URL': {'name': 'GRAFANA_API_URL', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_USERNAME': {'name': 'GRAFANA_API_USERNAME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_UPDATE_DASHBOARDS': {'name': 'GRAFANA_UPDATE_DASHBOARDS', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ISCSI_API_SSL_VERIFICATION': {'name': 'ISCSI_API_SSL_VERIFICATION', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'PROMETHEUS_API_HOST': {'name': 'PROMETHEUS_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'REST_REQUESTS_TIMEOUT': {'name': 'REST_REQUESTS_TIMEOUT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '45', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ACCESS_KEY': {'name': 'RGW_API_ACCESS_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ADMIN_RESOURCE': {'name': 'RGW_API_ADMIN_RESOURCE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_HOST': {'name': 'RGW_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_PORT': {'name': 'RGW_API_PORT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '80', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SCHEME': {'name': 'RGW_API_SCHEME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'http', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SECRET_KEY': {'name': 'RGW_API_SECRET_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SSL_VERIFY': {'name': 'RGW_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_USER_ID': {'name': 'RGW_API_USER_ID', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'crt_file': {'name': 'crt_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'debug': {'name': 'debug', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Enable/disable debug options', 'long_desc': '', 'tags': [], 'see_also': []}, 'jwt_token_ttl': {'name': 'jwt_token_ttl', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '28800', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '::', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8080', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl_server_port': {'name': 'ssl_server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8443', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_behaviour': {'name': 'standby_behaviour', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'redirect', 'min': '', 'max': '', 'enum_allowed': ['error', 'redirect'], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_error_status_code': {'name': 'standby_error_status_code', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '500', 'min': '400', 'max': '599', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url_prefix': {'name': 'url_prefix', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'deepsea', 'can_run': True, 'error_string': '', 'module_options': {'salt_api_eauth': {'name': 'salt_api_eauth', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'sharedsecret', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_password': {'name': 'salt_api_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_url': {'name': 'salt_api_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_username': {'name': 'salt_api_username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'devicehealth', 'can_run': True, 'error_string': '', 'module_options': {'enable_monitoring': {'name': 'enable_monitoring', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'monitor device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'mark_out_threshold': {'name': 'mark_out_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '2419200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically mark OSD if it may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_name': {'name': 'pool_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'device_health_metrics', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name of pool in which to store device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'retention_period': {'name': 'retention_period', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '15552000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_frequency': {'name': 'scrape_frequency', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to scrape device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'self_heal': {'name': 'self_heal', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'preemptively heal cluster around devices that may fail', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and check device health', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_threshold': {'name': 'warn_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '7257600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'raise health warning if OSD may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_cloud', 'can_run': True, 'error_string': '', 'module_options': {'diskprediction_cert_context': {'name': 'diskprediction_cert_context', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_default_authority': {'name': 'diskprediction_default_authority', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_password': {'name': 'diskprediction_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_port': {'name': 'diskprediction_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '31400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_retrieve_prediction_interval': {'name': 'diskprediction_retrieve_prediction_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_server': {'name': 'diskprediction_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_ssl_target_name_override': {'name': 'diskprediction_ssl_target_name_override', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_metrics_interval': {'name': 'diskprediction_upload_metrics_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_smart_interval': {'name': 'diskprediction_upload_smart_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_user': {'name': 'diskprediction_user', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_local', 'can_run': True, 'error_string': '', 'module_options': {'predict_interval': {'name': 'predict_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'influx', 'can_run': False, 'error_string': 'influxdb python module not found', 'module_options': {'batch_size': {'name': 'batch_size', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'database': {'name': 'database', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'hostname': {'name': 'hostname', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '30', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'port': {'name': 'port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '8086', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'false', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'threads': {'name': 'threads', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_ssl': {'name': 'verify_ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'true', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'insights', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'iostat', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'localpool', 'can_run': True, 'error_string': '', 'module_options': {'failure_domain': {'name': 'failure_domain', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'host', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'failure domain for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'min_size': {'name': 'min_size', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default min_size for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'num_rep': {'name': 'num_rep', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '3', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default replica count for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'pg_num': {'name': 'pg_num', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '128', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default pg_num for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'prefix': {'name': 'prefix', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name prefix for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'subtree': {'name': 'subtree', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'rack', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'CRUSH level for which to create a local pool', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'orchestrator_cli', 'can_run': True, 'error_string': '', 'module_options': {'orchestrator': {'name': 'orchestrator', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'pg_autoscaler', 'can_run': True, 'error_string': '', 'module_options': {'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'progress', 'can_run': True, 'error_string': '', 'module_options': {'max_completed_events': {'name': 'max_completed_events', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '50', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'number of past completed events to remember', 'long_desc': '', 'tags': [], 'see_also': []}, 'persist_interval': {'name': 'persist_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to persist completed events', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'prometheus', 'can_run': True, 'error_string': '', 'module_options': {'rbd_stats_pools': {'name': 'rbd_stats_pools', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rbd_stats_pools_refresh_interval': {'name': 'rbd_stats_pools_refresh_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_interval': {'name': 'scrape_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rbd_support', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'restful', 'can_run': True, 'error_string': '', 'module_options': {'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rook', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'selftest', 'can_run': True, 'error_string': '', 'module_options': {'roption1': {'name': 'roption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'roption2': {'name': 'roption2', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'xyz', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption1': {'name': 'rwoption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption2': {'name': 'rwoption2', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption3': {'name': 'rwoption3', 'type': 'float', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption4': {'name': 'rwoption4', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption5': {'name': 'rwoption5', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption6': {'name': 'rwoption6', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testkey': {'name': 'testkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testlkey': {'name': 'testlkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testnewline': {'name': 'testnewline', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ssh', 'can_run': True, 'error_string': '', 'module_options': {'inventory_cache_timeout_min': {'name': 'inventory_cache_timeout_min', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssh_config_file': {'name': 'ssh_config_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'status', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'telegraf', 'can_run': True, 'error_string': '', 'module_options': {'address': {'name': 'address', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'unixgram:///tmp/telegraf.sock', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '15', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'telemetry', 'can_run': True, 'error_string': '', 'module_options': {'channel_basic': {'name': 'channel_basic', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Share basic cluster information (size, version)', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_crash': {'name': 'channel_crash', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_device': {'name': 'channel_device', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_ident': {'name': 'channel_ident', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'contact': {'name': 'contact', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'description': {'name': 'description', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'device_url': {'name': 'device_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/device', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '24', 'min': '8', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'last_opt_revision': {'name': 'last_opt_revision', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '1', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'leaderboard': {'name': 'leaderboard', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'organization': {'name': 'organization', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'proxy': {'name': 'proxy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url': {'name': 'url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/report', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'test_orchestrator', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'volumes', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'zabbix', 'can_run': True, 'error_string': '', 'module_options': {'identifier': {'name': 'identifier', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_host': {'name': 'zabbix_host', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_port': {'name': 'zabbix_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '10051', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_sender': {'name': 'zabbix_sender', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '/usr/bin/zabbix_sender', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}], 'services': {}, 'always_on_modules': {'nautilus': ['balancer', 'crash', 'devicehealth', 'orchestrator_cli', 'progress', 'rbd_support', 'status', 'volumes']}}, 'servicemap': {'epoch': 1, 'modified': '0.000000', 'services': {}}, 'progress_events': {}} 2020-05-24T13:49:38.411 INFO:tasks.ceph.ceph_manager.c1:clean! 2020-05-24T13:49:38.411 INFO:tasks.ceph:Waiting until ceph cluster c1 is healthy... 2020-05-24T13:49:38.412 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 health 2020-05-24T13:49:39.288 INFO:teuthology.misc.health.smithi019.stdout:HEALTH_OK 2020-05-24T13:49:39.300 DEBUG:teuthology.misc:Ceph health: HEALTH_OK 2020-05-24T13:49:39.301 INFO:teuthology.run_tasks:Running task ceph... 2020-05-24T13:49:39.316 INFO:tasks.ceph:Creating ceph cluster c2... 2020-05-24T13:49:39.316 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'c2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-05-24T13:49:39.317 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087418', 'branch': 'nautilus', 'description': 'rgw/multisite/{clusters.yaml frontend/beast.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml}', 'email': 'tchaikov@gmail.com', 'first_in_suite': False, 'job_id': '5087418', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi', 'nuke-on-error': True, 'os_type': 'centos', 'overrides': {'admin_socket': {'branch': 'nautilus'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c1.client.1'], 'name': 'test-zone3'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone4'}]}]}, 'workunit': {'branch': 'nautilus-py3', 'sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff'}}, 'owner': 'scheduled_kchai@teuthology', 'priority': 90, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'nautilus-py3', 'suite_path': '/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/tchaikov/ceph.git', 'suite_sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}}, {'rgw': {'c1.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c1.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.27836', 'targets': {'smithi019.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnzKzvSvSSWhp8CQJE68ha/aIH8OK1pi/nONXFErfC78oXKR8ZNIhgv5FDzind/CpyduLhpPcJnw+zH2e3jwHhi0e6YEGVQEIBL8pr0+Sf0fZ9+jp4nBWKF0ZTbmy2hIidWRC4S7UCmFKYiZuyTa+FKyV0YcMB0ZRMa21Y73JVd3q01kJhd9i7F0PkutoM0SzMDC9xTmXxsH60VFXIs+um4giQplsiL5QZftzbKyyDw9ayPtMUOhR9AeI7YfmETAymVal3Xy5mzAPJIhmc0lPyQoDwYz1A2+e0VKUtb56uvbKvUBLS5qdoUZhxTLGhR80urGGYmCAeY+kza7R9eVkP', 'smithi193.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZVvdI2EmeM5Uc9ruLPP62O+Nj5/XERWhHEup3D2vviYzaLHVDcmOrC9OzuCxuSIi9SWsopYQUWD3ZekNEhKP0nqg2LVosDApxzBJ7c6gdkDMrwx6bm2uDVZVt/f6ghtUjnScO9SQeGFwpa5lkp3sxBBSkymUBAFklUc5qHzYdYpa3GhVtk7aeBOHN1de5nra+K2r94YhtOeSpXXKZybWPPenBWzRXZ8EoAkbgJPistYN1TAtCs6/54caTlA814tZ5akAMXihxO+qtUgdomjlJvasVtzTdrFHBEVhnQzt9Fd68/P9MicuvkffceblmK/n4xEjvWJttOVMAubyjtRvh'}} 2020-05-24T13:49:39.318 INFO:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2020-05-24T13:49:39.323 INFO:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2020-05-24T13:49:39.412 INFO:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:49:39.416 INFO:teuthology.orchestra.run.smithi193:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:49:39.616 DEBUG:teuthology.orchestra.remote:smithi193:/scratch_devs is 72B 2020-05-24T13:49:39.706 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-05-24T13:49:39.706 INFO:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-05-24T13:49:39.778 INFO:teuthology.orchestra.run.smithi193.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-0’ 2020-05-24T13:49:39.779 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:39.779 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 99929 Links: 1 2020-05-24T13:49:39.780 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:39.780 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:39.780 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2020-05-24 13:47:26.117467992 +0000 2020-05-24T13:49:39.780 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2020-05-24 13:45:46.961925048 +0000 2020-05-24T13:49:39.780 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2020-05-24 13:45:46.961925048 +0000 2020-05-24T13:49:39.780 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2020-05-24T13:49:39.803 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2020-05-24T13:49:39.803 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2020-05-24T13:49:39.803 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes (512 B) copied, 0.000342662 s, 1.5 MB/s 2020-05-24T13:49:39.806 INFO:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-05-24T13:49:39.885 INFO:teuthology.orchestra.run.smithi193.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-1’ 2020-05-24T13:49:39.885 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:39.885 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 99970 Links: 1 2020-05-24T13:49:39.886 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:39.886 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:39.886 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2020-05-24 13:47:26.117467992 +0000 2020-05-24T13:49:39.886 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2020-05-24 13:45:47.467917208 +0000 2020-05-24T13:49:39.887 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2020-05-24 13:45:47.467917208 +0000 2020-05-24T13:49:39.887 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2020-05-24T13:49:39.915 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2020-05-24T13:49:39.915 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2020-05-24T13:49:39.915 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes (512 B) copied, 0.000459593 s, 1.1 MB/s 2020-05-24T13:49:39.920 INFO:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-05-24T13:49:39.995 INFO:teuthology.orchestra.run.smithi193.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2020-05-24T13:49:39.996 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:39.996 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 106114 Links: 1 2020-05-24T13:49:39.996 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:39.997 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:39.997 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2020-05-24 13:47:26.118467992 +0000 2020-05-24T13:49:39.997 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2020-05-24 13:45:47.940909879 +0000 2020-05-24T13:49:39.997 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2020-05-24 13:45:47.940909879 +0000 2020-05-24T13:49:39.997 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2020-05-24T13:49:40.024 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2020-05-24T13:49:40.024 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2020-05-24T13:49:40.025 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes (512 B) copied, 0.000438963 s, 1.2 MB/s 2020-05-24T13:49:40.032 INFO:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-05-24T13:49:40.108 INFO:teuthology.orchestra.run.smithi193.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-3’ 2020-05-24T13:49:40.108 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:40.109 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 100025 Links: 1 2020-05-24T13:49:40.109 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:40.109 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:40.109 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2020-05-24 13:47:26.118467992 +0000 2020-05-24T13:49:40.110 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2020-05-24 13:45:48.396902814 +0000 2020-05-24T13:49:40.110 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2020-05-24 13:45:48.396902814 +0000 2020-05-24T13:49:40.110 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2020-05-24T13:49:40.135 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2020-05-24T13:49:40.135 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2020-05-24T13:49:40.136 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes (512 B) copied, 0.000406033 s, 1.3 MB/s 2020-05-24T13:49:40.140 INFO:tasks.ceph:osd dev map: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:40.142 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi193.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:49:40.142 INFO:tasks.ceph:Generating config... 2020-05-24T13:49:40.144 INFO:tasks.ceph:[client] debug rgw = 20 2020-05-24T13:49:40.144 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2020-05-24T13:49:40.144 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:49:40.144 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2020-05-24T13:49:40.145 INFO:tasks.ceph:[client] rgw data log num shards = 4 2020-05-24T13:49:40.145 INFO:tasks.ceph:[client] rgw md log max shards = 4 2020-05-24T13:49:40.145 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2020-05-24T13:49:40.146 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2020-05-24T13:49:40.146 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-05-24T13:49:40.146 INFO:tasks.ceph:[mgr] debug ms = 1 2020-05-24T13:49:40.147 INFO:tasks.ceph:[mon] debug mon = 20 2020-05-24T13:49:40.147 INFO:tasks.ceph:[mon] debug ms = 1 2020-05-24T13:49:40.147 INFO:tasks.ceph:[mon] debug paxos = 20 2020-05-24T13:49:40.147 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2020-05-24T13:49:40.148 INFO:tasks.ceph:[osd] debug filestore = 20 2020-05-24T13:49:40.148 INFO:tasks.ceph:[osd] debug journal = 20 2020-05-24T13:49:40.148 INFO:tasks.ceph:[osd] debug ms = 20 2020-05-24T13:49:40.149 INFO:tasks.ceph:[osd] debug osd = 25 2020-05-24T13:49:40.149 INFO:tasks.ceph:[osd] osd fast shutdown = False 2020-05-24T13:49:40.149 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2020-05-24T13:49:40.149 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2020-05-24T13:49:40.150 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2020-05-24T13:49:40.150 INFO:tasks.ceph:Setting up c2.mon.a... 2020-05-24T13:49:40.151 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c2.keyring 2020-05-24T13:49:40.336 INFO:teuthology.orchestra.run.smithi193.stdout:creating /etc/ceph/c2.keyring 2020-05-24T13:49:40.341 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c2.keyring 2020-05-24T13:49:40.507 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/c2.keyring 2020-05-24T13:49:40.619 DEBUG:tasks.ceph:Ceph mon addresses: [('c2.mon.a', '172.21.15.193')] 2020-05-24T13:49:40.619 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.193 --print /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:40.774 INFO:teuthology.orchestra.run.smithi193.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:40.774 INFO:teuthology.orchestra.run.smithi193.stdout:monmaptool: generated fsid be042078-2e26-4e73-a3c1-b7d9ac081979 2020-05-24T13:49:40.774 INFO:teuthology.orchestra.run.smithi193.stdout:epoch 0 2020-05-24T13:49:40.775 INFO:teuthology.orchestra.run.smithi193.stdout:fsid be042078-2e26-4e73-a3c1-b7d9ac081979 2020-05-24T13:49:40.775 INFO:teuthology.orchestra.run.smithi193.stdout:last_changed 2020-05-24 13:49:40.775144 2020-05-24T13:49:40.775 INFO:teuthology.orchestra.run.smithi193.stdout:created 2020-05-24 13:49:40.775144 2020-05-24T13:49:40.775 INFO:teuthology.orchestra.run.smithi193.stdout:min_mon_release 0 (unknown) 2020-05-24T13:49:40.776 INFO:teuthology.orchestra.run.smithi193.stdout:0: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.a 2020-05-24T13:49:40.776 INFO:teuthology.orchestra.run.smithi193.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c2.monmap (1 monitors) 2020-05-24T13:49:40.777 INFO:tasks.ceph:Writing /etc/ceph/c2.conf for FSID be042078-2e26-4e73-a3c1-b7d9ac081979... 2020-05-24T13:49:40.780 INFO:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2020-05-24T13:49:40.784 INFO:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2020-05-24T13:49:40.926 INFO:teuthology.orchestra.run.smithi193.stdout:[global] 2020-05-24T13:49:40.927 INFO:teuthology.orchestra.run.smithi193.stdout: chdir = "" 2020-05-24T13:49:40.927 INFO:teuthology.orchestra.run.smithi193.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:49:40.927 INFO:teuthology.orchestra.run.smithi193.stdout: auth supported = cephx 2020-05-24T13:49:40.928 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.928 INFO:teuthology.orchestra.run.smithi193.stdout: filestore xattr use omap = true 2020-05-24T13:49:40.929 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.929 INFO:teuthology.orchestra.run.smithi193.stdout: mon clock drift allowed = 1.000 2020-05-24T13:49:40.930 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.930 INFO:teuthology.orchestra.run.smithi193.stdout: osd crush chooseleaf type = 0 2020-05-24T13:49:40.930 INFO:teuthology.orchestra.run.smithi193.stdout: auth debug = true 2020-05-24T13:49:40.931 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.931 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on old message = true 2020-05-24T13:49:40.931 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on bug = true 2020-05-24T13:49:40.932 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.932 INFO:teuthology.orchestra.run.smithi193.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:49:40.932 INFO:teuthology.orchestra.run.smithi193.stdout: mon pg warn max object skew = 0 2020-05-24T13:49:40.932 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.933 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default size = 2 2020-05-24T13:49:40.933 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.933 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow primary affinity = true 2020-05-24T13:49:40.934 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow pg remap = true 2020-05-24T13:49:40.934 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:49:40.934 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:49:40.935 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on no sortbitwise = false 2020-05-24T13:49:40.935 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:49:40.935 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on too few osds = false 2020-05-24T13:49:40.936 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:49:40.936 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:49:40.936 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.936 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:49:40.937 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.937 INFO:teuthology.orchestra.run.smithi193.stdout: osd default data pool replay window = 5 2020-05-24T13:49:40.937 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.937 INFO:teuthology.orchestra.run.smithi193.stdout: mon allow pool delete = true 2020-05-24T13:49:40.938 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.938 INFO:teuthology.orchestra.run.smithi193.stdout: mon cluster log file level = debug 2020-05-24T13:49:40.938 INFO:teuthology.orchestra.run.smithi193.stdout: debug asserts on shutdown = true 2020-05-24T13:49:40.939 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.939 INFO:teuthology.orchestra.run.smithi193.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:49:40.939 INFO:teuthology.orchestra.run.smithi193.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:49:40.950 INFO:teuthology.orchestra.run.smithi193.stdout: mon host = 172.21.15.193 2020-05-24T13:49:40.950 INFO:teuthology.orchestra.run.smithi193.stdout: osd heartbeat grace = 40 2020-05-24T13:49:40.950 INFO:teuthology.orchestra.run.smithi193.stdout: fsid = be042078-2e26-4e73-a3c1-b7d9ac081979 2020-05-24T13:49:40.950 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.951 INFO:teuthology.orchestra.run.smithi193.stdout:[osd] 2020-05-24T13:49:40.951 INFO:teuthology.orchestra.run.smithi193.stdout: osd journal size = 100 2020-05-24T13:49:40.951 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.951 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub load threshold = 5.0 2020-05-24T13:49:40.952 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub max interval = 600 2020-05-24T13:49:40.952 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.952 INFO:teuthology.orchestra.run.smithi193.stdout: osd recover clone overlap = true 2020-05-24T13:49:40.953 INFO:teuthology.orchestra.run.smithi193.stdout: osd recovery max chunk = 1048576 2020-05-24T13:49:40.953 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.953 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug shutdown = true 2020-05-24T13:49:40.954 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug op order = true 2020-05-24T13:49:40.954 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug verify stray on activate = true 2020-05-24T13:49:40.954 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.954 INFO:teuthology.orchestra.run.smithi193.stdout: osd open classes on start = true 2020-05-24T13:49:40.955 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug pg log writeout = true 2020-05-24T13:49:40.955 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.955 INFO:teuthology.orchestra.run.smithi193.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:49:40.955 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.955 INFO:teuthology.orchestra.run.smithi193.stdout: osd map max advance = 10 2020-05-24T13:49:40.956 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.956 INFO:teuthology.orchestra.run.smithi193.stdout: journal zero on create = true 2020-05-24T13:49:40.956 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.956 INFO:teuthology.orchestra.run.smithi193.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:49:40.956 INFO:teuthology.orchestra.run.smithi193.stdout: filestore apply finisher threads = 3 2020-05-24T13:49:40.957 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.957 INFO:teuthology.orchestra.run.smithi193.stdout: bdev debug aio = true 2020-05-24T13:49:40.957 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug misdirected ops = true 2020-05-24T13:49:40.957 INFO:teuthology.orchestra.run.smithi193.stdout: debug filestore = 20 2020-05-24T13:49:40.957 INFO:teuthology.orchestra.run.smithi193.stdout: debug journal = 20 2020-05-24T13:49:40.958 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 20 2020-05-24T13:49:40.958 INFO:teuthology.orchestra.run.smithi193.stdout: debug osd = 25 2020-05-24T13:49:40.958 INFO:teuthology.orchestra.run.smithi193.stdout: osd fast shutdown = False 2020-05-24T13:49:40.958 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.958 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr] 2020-05-24T13:49:40.959 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2020-05-24T13:49:40.959 INFO:teuthology.orchestra.run.smithi193.stdout: debug mgr = 20 2020-05-24T13:49:40.959 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2020-05-24T13:49:40.959 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2020-05-24T13:49:40.959 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:40.960 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:40.960 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.960 INFO:teuthology.orchestra.run.smithi193.stdout:[mon] 2020-05-24T13:49:40.960 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2020-05-24T13:49:40.961 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2020-05-24T13:49:40.961 INFO:teuthology.orchestra.run.smithi193.stdout: debug paxos = 20 2020-05-24T13:49:40.961 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2020-05-24T13:49:40.961 INFO:teuthology.orchestra.run.smithi193.stdout: mon data avail warn = 5 2020-05-24T13:49:40.970 INFO:teuthology.orchestra.run.smithi193.stdout: mon mgr mkfs grace = 240 2020-05-24T13:49:40.970 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:40.970 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd reporter subtree level = osd 2020-05-24T13:49:40.971 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd prime pg temp = true 2020-05-24T13:49:40.971 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:40.971 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd crush smoke test = False 2020-05-24T13:49:40.971 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:49:40.971 INFO:teuthology.orchestra.run.smithi193.stdout:[client] 2020-05-24T13:49:40.972 INFO:teuthology.orchestra.run.smithi193.stdout: rgw cache enabled = true 2020-05-24T13:49:40.972 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable ops log = true 2020-05-24T13:49:40.972 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable usage log = true 2020-05-24T13:49:40.972 INFO:teuthology.orchestra.run.smithi193.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:49:40.972 INFO:teuthology.orchestra.run.smithi193.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:49:40.972 INFO:teuthology.orchestra.run.smithi193.stdout: debug rgw = 20 2020-05-24T13:49:40.973 INFO:teuthology.orchestra.run.smithi193.stdout: rgw crypt require ssl = False 2020-05-24T13:49:40.973 INFO:teuthology.orchestra.run.smithi193.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:49:40.973 INFO:teuthology.orchestra.run.smithi193.stdout: rgw curl low speed time = 300 2020-05-24T13:49:40.973 INFO:teuthology.orchestra.run.smithi193.stdout: rgw data log num shards = 4 2020-05-24T13:49:40.973 INFO:teuthology.orchestra.run.smithi193.stdout: rgw md log max shards = 4 2020-05-24T13:49:40.974 INFO:teuthology.orchestra.run.smithi193.stdout: rgw sync log trim interval = 0 2020-05-24T13:49:40.974 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.a] 2020-05-24T13:49:40.974 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.0] 2020-05-24T13:49:40.974 INFO:teuthology.orchestra.run.smithi193.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:49:40.974 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.1] 2020-05-24T13:49:40.975 INFO:teuthology.orchestra.run.smithi193.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:49:40.975 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.2] 2020-05-24T13:49:40.975 INFO:teuthology.orchestra.run.smithi193.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:49:40.976 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2020-05-24T13:49:40.977 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2020-05-24T13:49:40.977 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:49:40.977 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2020-05-24T13:49:40.977 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.978 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2020-05-24T13:49:40.978 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.978 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2020-05-24T13:49:40.978 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.978 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2020-05-24T13:49:40.978 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2020-05-24T13:49:40.979 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.979 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2020-05-24T13:49:40.979 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2020-05-24T13:49:40.979 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.979 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:49:40.980 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2020-05-24T13:49:40.980 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.980 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2020-05-24T13:49:40.980 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.980 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2020-05-24T13:49:40.981 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2020-05-24T13:49:40.981 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:49:40.981 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:49:40.981 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2020-05-24T13:49:40.981 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:49:40.981 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2020-05-24T13:49:40.982 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:49:40.982 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:49:40.982 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.982 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:49:40.982 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.983 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2020-05-24T13:49:40.983 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.983 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2020-05-24T13:49:40.983 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.983 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2020-05-24T13:49:40.983 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2020-05-24T13:49:40.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.984 INFO:teuthology.orchestra.run.smithi019.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:49:40.984 INFO:teuthology.orchestra.run.smithi019.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:49:40.984 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = 172.21.15.193 2020-05-24T13:49:40.984 INFO:teuthology.orchestra.run.smithi019.stdout: osd heartbeat grace = 40 2020-05-24T13:49:40.985 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = be042078-2e26-4e73-a3c1-b7d9ac081979 2020-05-24T13:49:40.985 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.985 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2020-05-24T13:49:40.985 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2020-05-24T13:49:40.985 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.985 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2020-05-24T13:49:40.986 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2020-05-24T13:49:40.986 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.986 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2020-05-24T13:49:40.986 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2020-05-24T13:49:40.986 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.987 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2020-05-24T13:49:40.987 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2020-05-24T13:49:40.987 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2020-05-24T13:49:40.987 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.987 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2020-05-24T13:49:40.987 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2020-05-24T13:49:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:49:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2020-05-24T13:49:40.988 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.989 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2020-05-24T13:49:40.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.989 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:49:40.989 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2020-05-24T13:49:40.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.989 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2020-05-24T13:49:40.990 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2020-05-24T13:49:40.990 INFO:teuthology.orchestra.run.smithi019.stdout: debug filestore = 20 2020-05-24T13:49:40.990 INFO:teuthology.orchestra.run.smithi019.stdout: debug journal = 20 2020-05-24T13:49:40.990 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 20 2020-05-24T13:49:40.990 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 25 2020-05-24T13:49:40.991 INFO:teuthology.orchestra.run.smithi019.stdout: osd fast shutdown = False 2020-05-24T13:49:40.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.991 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2020-05-24T13:49:40.991 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2020-05-24T13:49:40.991 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2020-05-24T13:49:40.991 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2020-05-24T13:49:40.992 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2020-05-24T13:49:40.992 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:40.992 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:40.992 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.992 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2020-05-24T13:49:40.993 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2020-05-24T13:49:40.993 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2020-05-24T13:49:40.993 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2020-05-24T13:49:40.993 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2020-05-24T13:49:40.993 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2020-05-24T13:49:40.993 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2020-05-24T13:49:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2020-05-24T13:49:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2020-05-24T13:49:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd crush smoke test = False 2020-05-24T13:49:40.995 INFO:teuthology.orchestra.run.smithi019.stdout: 2020-05-24T13:49:40.995 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2020-05-24T13:49:40.995 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2020-05-24T13:49:40.995 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2020-05-24T13:49:40.995 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2020-05-24T13:49:40.995 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:49:40.996 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:49:40.996 INFO:teuthology.orchestra.run.smithi019.stdout: debug rgw = 20 2020-05-24T13:49:40.996 INFO:teuthology.orchestra.run.smithi019.stdout: rgw crypt require ssl = False 2020-05-24T13:49:40.996 INFO:teuthology.orchestra.run.smithi019.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:49:40.996 INFO:teuthology.orchestra.run.smithi019.stdout: rgw curl low speed time = 300 2020-05-24T13:49:40.997 INFO:teuthology.orchestra.run.smithi019.stdout: rgw data log num shards = 4 2020-05-24T13:49:40.997 INFO:teuthology.orchestra.run.smithi019.stdout: rgw md log max shards = 4 2020-05-24T13:49:40.997 INFO:teuthology.orchestra.run.smithi019.stdout: rgw sync log trim interval = 0 2020-05-24T13:49:40.997 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2020-05-24T13:49:40.997 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2020-05-24T13:49:40.998 INFO:teuthology.orchestra.run.smithi019.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:49:40.998 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2020-05-24T13:49:40.998 INFO:teuthology.orchestra.run.smithi019.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:49:40.998 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2020-05-24T13:49:40.998 INFO:teuthology.orchestra.run.smithi019.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:49:40.999 INFO:tasks.ceph:Creating admin key on c2.mon.a... 2020-05-24T13:49:40.999 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c2.keyring 2020-05-24T13:49:41.197 INFO:tasks.ceph:Copying monmap to all nodes... 2020-05-24T13:49:41.284 DEBUG:teuthology.orchestra.remote:smithi193:/etc/ceph/c2.keyring is 206B 2020-05-24T13:49:41.437 DEBUG:teuthology.orchestra.remote:smithi193:/home/ubuntu/cephtest/c2.monmap is 177B 2020-05-24T13:49:41.523 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2020-05-24T13:49:41.524 INFO:teuthology.orchestra.run.smithi019:> sudo sh -c 'cat > /etc/ceph/c2.keyring' && sudo chmod 0644 /etc/ceph/c2.keyring 2020-05-24T13:49:41.682 INFO:teuthology.orchestra.run.smithi019:> cat > /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:41.763 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:49:41.764 INFO:teuthology.orchestra.run.smithi193:> sudo sh -c 'cat > /etc/ceph/c2.keyring' && sudo chmod 0644 /etc/ceph/c2.keyring 2020-05-24T13:49:41.912 INFO:teuthology.orchestra.run.smithi193:> cat > /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:41.976 INFO:tasks.ceph:Setting up mon nodes... 2020-05-24T13:49:41.977 INFO:tasks.ceph:Setting up mgr nodes... 2020-05-24T13:49:41.978 INFO:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mgr/c2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c2-x/keyring 2020-05-24T13:49:42.208 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mgr/c2-x/keyring 2020-05-24T13:49:42.213 INFO:tasks.ceph:Setting up mds nodes... 2020-05-24T13:49:42.214 INFO:tasks.ceph_client:Setting up client nodes... 2020-05-24T13:49:42.214 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c2.client.0.keyring && sudo chmod 0644 /etc/ceph/c2.client.0.keyring 2020-05-24T13:49:42.348 INFO:teuthology.orchestra.run.smithi193.stdout:creating /etc/ceph/c2.client.0.keyring 2020-05-24T13:49:42.376 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c2.client.1.keyring && sudo chmod 0644 /etc/ceph/c2.client.1.keyring 2020-05-24T13:49:42.506 INFO:teuthology.orchestra.run.smithi193.stdout:creating /etc/ceph/c2.client.1.keyring 2020-05-24T13:49:42.532 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-05-24T13:49:42.532 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi193.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:49:42.533 INFO:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/c2-0 2020-05-24T13:49:42.655 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:42.655 INFO:tasks.ceph:role: c2.osd.0 2020-05-24T13:49:42.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:49:42.656 INFO:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-05-24T13:49:43.255 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:49:43.383 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:49:43.384 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:49:43.384 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:49:43.384 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:49:43.384 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:49:43.384 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:49:43.385 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:49:43.385 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:49:43.385 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2020-05-24T13:49:43.386 INFO:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c2-0 2020-05-24T13:49:43.588 INFO:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-0 2020-05-24T13:49:43.697 INFO:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/c2-1 2020-05-24T13:49:43.780 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:43.780 INFO:tasks.ceph:role: c2.osd.1 2020-05-24T13:49:43.781 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:49:43.781 INFO:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-05-24T13:49:44.348 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:49:44.348 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:49:44.348 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:49:44.348 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:49:44.349 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:49:44.349 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:49:44.349 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:49:44.349 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:49:44.349 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:49:44.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2020-05-24T13:49:44.351 INFO:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c2-1 2020-05-24T13:49:44.490 INFO:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-1 2020-05-24T13:49:44.564 INFO:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/c2-2 2020-05-24T13:49:44.637 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:44.638 INFO:tasks.ceph:role: c2.osd.2 2020-05-24T13:49:44.638 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi193.front.sepia.ceph.com 2020-05-24T13:49:44.638 INFO:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-05-24T13:49:45.177 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:49:45.177 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:49:45.177 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:49:45.178 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:49:45.178 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:49:45.178 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:49:45.178 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:49:45.179 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:49:45.180 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:49:45.181 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2020-05-24T13:49:45.181 INFO:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c2-2 2020-05-24T13:49:45.340 INFO:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-2 2020-05-24T13:49:45.474 INFO:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:45.677 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:45.675 7fca9a6ba9c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-0/keyring: can't open /var/lib/ceph/osd/c2-0/keyring: (2) No such file or directory 2020-05-24T13:49:45.680 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:45.680 7fca9a6ba9c0 -1 created new key in keyring /var/lib/ceph/osd/c2-0/keyring 2020-05-24T13:49:45.681 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:45.680 7fca9a6ba9c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2020-05-24T13:49:45.681 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:45.680 7fca9a6ba9c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2020-05-24T13:49:45.681 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:45.680 7fca9a6ba9c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2020-05-24T13:49:45.688 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:45.688 7fca9a6ba9c0 -1 bluestore(/var/lib/ceph/osd/c2-0) _read_fsid unparsable uuid 2020-05-24T13:49:47.737 INFO:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-0 2020-05-24T13:49:47.856 INFO:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:48.047 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:48.047 7f942bfef9c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-1/keyring: can't open /var/lib/ceph/osd/c2-1/keyring: (2) No such file or directory 2020-05-24T13:49:48.048 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:48.048 7f942bfef9c0 -1 created new key in keyring /var/lib/ceph/osd/c2-1/keyring 2020-05-24T13:49:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:48.048 7f942bfef9c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2020-05-24T13:49:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:48.048 7f942bfef9c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2020-05-24T13:49:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:48.048 7f942bfef9c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2020-05-24T13:49:48.050 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:48.049 7f942bfef9c0 -1 bluestore(/var/lib/ceph/osd/c2-1) _read_fsid unparsable uuid 2020-05-24T13:49:50.101 INFO:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-1 2020-05-24T13:49:50.211 INFO:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:50.384 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:50.383 7fac0259b9c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-2/keyring: can't open /var/lib/ceph/osd/c2-2/keyring: (2) No such file or directory 2020-05-24T13:49:50.384 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:50.383 7fac0259b9c0 -1 created new key in keyring /var/lib/ceph/osd/c2-2/keyring 2020-05-24T13:49:50.384 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:50.384 7fac0259b9c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2020-05-24T13:49:50.385 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:50.384 7fac0259b9c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2020-05-24T13:49:50.385 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:50.384 7fac0259b9c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2020-05-24T13:49:50.385 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:49:50.385 7fac0259b9c0 -1 bluestore(/var/lib/ceph/osd/c2-2) _read_fsid unparsable uuid 2020-05-24T13:49:52.434 INFO:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-2 2020-05-24T13:49:52.546 INFO:tasks.ceph:Reading keys from all nodes... 2020-05-24T13:49:52.547 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:49:52.634 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.2Ik5gjCefm 2020-05-24T13:49:52.634 INFO:teuthology.orchestra.run.smithi193:> sudo cp /var/lib/ceph/mgr/c2-x/keyring /tmp/tmp.2Ik5gjCefm 2020-05-24T13:49:52.746 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.2Ik5gjCefm 2020-05-24T13:49:52.912 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.2Ik5gjCefm is 56B 2020-05-24T13:49:52.982 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.2Ik5gjCefm 2020-05-24T13:49:53.043 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:49:53.100 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.NCOZUlxgtW 2020-05-24T13:49:53.101 INFO:teuthology.orchestra.run.smithi193:> sudo cp /var/lib/ceph/osd/c2-0/keyring /tmp/tmp.NCOZUlxgtW 2020-05-24T13:49:53.174 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.NCOZUlxgtW 2020-05-24T13:49:53.289 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.NCOZUlxgtW is 56B 2020-05-24T13:49:53.345 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.NCOZUlxgtW 2020-05-24T13:49:53.392 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:49:53.512 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.YYZb0AnVPm 2020-05-24T13:49:53.513 INFO:teuthology.orchestra.run.smithi193:> sudo cp /var/lib/ceph/osd/c2-1/keyring /tmp/tmp.YYZb0AnVPm 2020-05-24T13:49:53.611 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.YYZb0AnVPm 2020-05-24T13:49:53.792 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.YYZb0AnVPm is 56B 2020-05-24T13:49:53.857 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.YYZb0AnVPm 2020-05-24T13:49:53.939 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:49:54.059 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.hkDaMbM5lk 2020-05-24T13:49:54.060 INFO:teuthology.orchestra.run.smithi193:> sudo cp /var/lib/ceph/osd/c2-2/keyring /tmp/tmp.hkDaMbM5lk 2020-05-24T13:49:54.171 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.hkDaMbM5lk 2020-05-24T13:49:54.363 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.hkDaMbM5lk is 56B 2020-05-24T13:49:54.446 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.hkDaMbM5lk 2020-05-24T13:49:54.615 DEBUG:teuthology.orchestra.remote:smithi193:/etc/ceph/c2.client.0.keyring is 59B 2020-05-24T13:49:54.778 DEBUG:teuthology.orchestra.remote:smithi193:/etc/ceph/c2.client.1.keyring is 59B 2020-05-24T13:49:54.857 INFO:tasks.ceph:Adding keys to all mons... 2020-05-24T13:49:54.858 INFO:teuthology.orchestra.run.smithi193:> sudo tee -a /etc/ceph/c2.keyring 2020-05-24T13:49:54.942 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr.x] 2020-05-24T13:49:54.942 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQB2e8peE1J8DBAAW3NLKvUx9sZLk5sZ3wPbfg== 2020-05-24T13:49:54.942 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.0] 2020-05-24T13:49:54.943 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQB5e8pe8pxfKBAAkHnz1Do2LYaQ3c/Hu6Xl7w== 2020-05-24T13:49:54.943 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.1] 2020-05-24T13:49:54.943 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQB8e8pe9kHpAhAAspYmyCXUkyQg2hGCwuBfPA== 2020-05-24T13:49:54.943 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.2] 2020-05-24T13:49:54.943 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQB+e8peSaztFhAAnM01VwPFxAHjw7BeGSigLA== 2020-05-24T13:49:54.944 INFO:teuthology.orchestra.run.smithi193.stdout:[client.0] 2020-05-24T13:49:54.944 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQB2e8peD3PUFBAAKxF1+DGJI6g7reeLSRisYQ== 2020-05-24T13:49:54.944 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2020-05-24T13:49:54.944 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQB2e8pe9B07HhAAFp5uv/h4HyYCiILhz222Mw== 2020-05-24T13:49:54.945 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-24T13:49:55.084 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:49:55.209 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:49:55.353 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:49:55.519 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:49:55.702 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:49:55.877 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-05-24T13:49:55.877 INFO:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mon/c2-a 2020-05-24T13:49:55.989 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c2 --mkfs -i a --monmap /home/ubuntu/cephtest/c2.monmap --keyring /etc/ceph/c2.keyring 2020-05-24T13:49:56.157 INFO:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c2-a 2020-05-24T13:49:56.252 INFO:teuthology.orchestra.run.smithi193:> rm -- /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:56.320 INFO:tasks.ceph:Starting mon daemons in cluster c2... 2020-05-24T13:49:56.321 DEBUG:teuthology.misc:running c2.mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2020-05-24T13:49:56.322 INFO:tasks.ceph.c2.mon.a:Restarting daemon 2020-05-24T13:49:56.323 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.mon.a.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster c2 -i a 2020-05-24T13:49:56.327 INFO:tasks.ceph.c2.mon.a:Started 2020-05-24T13:49:56.328 INFO:tasks.ceph:Starting mgr daemons in cluster c2... 2020-05-24T13:49:56.329 INFO:tasks.ceph.c2.mgr.x:Restarting daemon 2020-05-24T13:49:56.329 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster c2 -i x 2020-05-24T13:49:56.333 INFO:tasks.ceph.c2.mgr.x:Started 2020-05-24T13:49:56.333 INFO:tasks.ceph:Setting crush tunables to default 2020-05-24T13:49:56.334 INFO:teuthology.orchestra.run.smithi193:> sudo ceph --cluster c2 osd crush tunables default 2020-05-24T13:50:01.866 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:00:05.051 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T13:50:07.356 INFO:teuthology.orchestra.run.smithi193.stderr:adjusted tunables profile to default 2020-05-24T13:50:07.375 INFO:tasks.ceph:Starting osd daemons in cluster c2... 2020-05-24T13:50:07.376 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:50:07.454 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.VzCYDso4zI 2020-05-24T13:50:07.454 INFO:teuthology.orchestra.run.smithi193:> sudo cp /var/lib/ceph/osd/c2-0/fsid /tmp/tmp.VzCYDso4zI 2020-05-24T13:50:07.524 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.VzCYDso4zI 2020-05-24T13:50:07.624 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.VzCYDso4zI is 37B 2020-05-24T13:50:07.671 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.VzCYDso4zI 2020-05-24T13:50:07.717 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:50:07.810 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.Hch0v8tT3o 2020-05-24T13:50:07.811 INFO:teuthology.orchestra.run.smithi193:> sudo cp /var/lib/ceph/osd/c2-1/fsid /tmp/tmp.Hch0v8tT3o 2020-05-24T13:50:07.883 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.Hch0v8tT3o 2020-05-24T13:50:07.993 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.Hch0v8tT3o is 37B 2020-05-24T13:50:08.039 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.Hch0v8tT3o 2020-05-24T13:50:08.080 INFO:teuthology.orchestra.run.smithi193:> mktemp 2020-05-24T13:50:08.174 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.jqWoEZmLa6 2020-05-24T13:50:08.175 INFO:teuthology.orchestra.run.smithi193:> sudo cp /var/lib/ceph/osd/c2-2/fsid /tmp/tmp.jqWoEZmLa6 2020-05-24T13:50:08.244 INFO:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.jqWoEZmLa6 2020-05-24T13:50:08.350 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.jqWoEZmLa6 is 37B 2020-05-24T13:50:08.399 INFO:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.jqWoEZmLa6 2020-05-24T13:50:08.441 INFO:teuthology.orchestra.run.smithi193:> sudo ceph --cluster c2 osd new 9273eda0-c3ce-4aa8-976b-bd91d636f366 0 2020-05-24T13:50:09.456 INFO:teuthology.orchestra.run.smithi193.stdout:0 2020-05-24T13:50:09.479 INFO:teuthology.orchestra.run.smithi193:> sudo ceph --cluster c2 osd new 72d8fe1d-7d45-41c3-9a3a-c83cb3afee51 1 2020-05-24T13:50:10.479 INFO:teuthology.orchestra.run.smithi193.stdout:1 2020-05-24T13:50:10.503 INFO:teuthology.orchestra.run.smithi193:> sudo ceph --cluster c2 osd new 7c234f2d-8854-4e13-af2b-8c5d113e1824 2 2020-05-24T13:50:11.523 INFO:teuthology.orchestra.run.smithi193.stdout:2 2020-05-24T13:50:11.547 DEBUG:teuthology.misc:running c2.osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:50:11.547 INFO:tasks.ceph.c2.osd.0:Restarting daemon 2020-05-24T13:50:11.548 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.osd.0.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c2 -i 0 2020-05-24T13:50:11.551 INFO:tasks.ceph.c2.osd.0:Started 2020-05-24T13:50:11.551 DEBUG:teuthology.misc:running c2.osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:50:11.552 INFO:tasks.ceph.c2.osd.1:Restarting daemon 2020-05-24T13:50:11.552 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.osd.1.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c2 -i 1 2020-05-24T13:50:11.555 INFO:tasks.ceph.c2.osd.1:Started 2020-05-24T13:50:11.555 DEBUG:teuthology.misc:running c2.osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:50:11.556 INFO:tasks.ceph.c2.osd.2:Restarting daemon 2020-05-24T13:50:11.556 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.osd.2.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c2 -i 2 2020-05-24T13:50:11.559 INFO:tasks.ceph.c2.osd.2:Started 2020-05-24T13:50:11.560 INFO:tasks.ceph:Waiting for OSDs to come up 2020-05-24T13:50:11.862 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:12.712 INFO:teuthology.misc.health.smithi193.stdout: 2020-05-24T13:50:12.712 INFO:teuthology.misc.health.smithi193.stdout:{"epoch":5,"fsid":"be042078-2e26-4e73-a3c1-b7d9ac081979","created":"2020-05-24 13:50:04.563678","modified":"2020-05-24 13:50:11.417785","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"9273eda0-c3ce-4aa8-976b-bd91d636f366","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"72d8fe1d-7d45-41c3-9a3a-c83cb3afee51","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"7c234f2d-8854-4e13-af2b-8c5d113e1824","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:12.723 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:50:17.956 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:==00:00:00:06.263 11120== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:17.971 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:==00:00:00:06.268 11121== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:17.972 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:==00:00:00:06.287 11119== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:18.014 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:==00:00:00:06.321 11120== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:18.026 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:==00:00:00:06.341 11119== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:18.030 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:==00:00:00:06.327 11121== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:18.616 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:2020-05-24 13:50:18.598 9ae5e40 -1 Falling back to public interface 2020-05-24T13:50:18.627 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:2020-05-24 13:50:18.610 9ae5e40 -1 Falling back to public interface 2020-05-24T13:50:18.632 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:2020-05-24 13:50:18.611 9ae5e40 -1 Falling back to public interface 2020-05-24T13:50:18.851 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:==00:00:00:07.158 11120== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:18.856 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:==00:00:00:07.153 11121== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:18.884 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:==00:00:00:07.199 11119== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:19.027 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:19.228 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:==00:00:00:07.535 11120== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:19.237 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:==00:00:00:07.543 11120== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:19.253 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:==00:00:00:07.550 11121== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:19.262 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:==00:00:00:07.559 11121== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:19.270 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:==00:00:00:07.584 11119== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:19.278 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:==00:00:00:07.593 11119== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:19.767 INFO:teuthology.misc.health.smithi193.stdout: 2020-05-24T13:50:19.768 INFO:teuthology.misc.health.smithi193.stdout:{"epoch":5,"fsid":"be042078-2e26-4e73-a3c1-b7d9ac081979","created":"2020-05-24 13:50:04.563678","modified":"2020-05-24 13:50:11.417785","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"9273eda0-c3ce-4aa8-976b-bd91d636f366","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"72d8fe1d-7d45-41c3-9a3a-c83cb3afee51","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"7c234f2d-8854-4e13-af2b-8c5d113e1824","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:19.779 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:50:20.111 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:==00:00:00:08.417 11120== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:20.144 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:==00:00:00:08.440 11121== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:20.175 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:==00:00:00:08.489 11119== Warning: unimplemented fcntl command: 1038 2020-05-24T13:50:21.467 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:2020-05-24 13:50:21.463 9ae5e40 -1 osd.1 0 log_to_monitors {default=true} 2020-05-24T13:50:21.495 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:2020-05-24 13:50:21.488 9ae5e40 -1 osd.2 0 log_to_monitors {default=true} 2020-05-24T13:50:21.739 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:2020-05-24 13:50:21.732 9ae5e40 -1 osd.0 0 log_to_monitors {default=true} 2020-05-24T13:50:23.219 INFO:tasks.ceph.c2.osd.2.smithi193.stderr:2020-05-24 13:50:23.168 290bd700 -1 osd.2 0 waiting for initial osdmap 2020-05-24T13:50:23.390 INFO:tasks.ceph.c2.osd.1.smithi193.stderr:2020-05-24 13:50:23.170 28cbc700 -1 osd.1 0 waiting for initial osdmap 2020-05-24T13:50:24.370 INFO:tasks.ceph.c2.osd.0.smithi193.stderr:2020-05-24 13:50:24.364 28cbc700 -1 osd.0 0 waiting for initial osdmap 2020-05-24T13:50:26.082 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:27.019 INFO:teuthology.misc.health.smithi193.stdout: 2020-05-24T13:50:27.020 INFO:teuthology.misc.health.smithi193.stdout:{"epoch":9,"fsid":"be042078-2e26-4e73-a3c1-b7d9ac081979","created":"2020-05-24 13:50:04.563678","modified":"2020-05-24 13:50:25.160755","last_up_change":"2020-05-24 13:50:25.160755","last_in_change":"2020-05-24 13:50:25.160755","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"9273eda0-c3ce-4aa8-976b-bd91d636f366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":11119},{"type":"v1","addr":"172.21.15.193:6808","nonce":11119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6813","nonce":11119},{"type":"v1","addr":"172.21.15.193:6815","nonce":11119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":11119},{"type":"v1","addr":"172.21.15.193:6825","nonce":11119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6819","nonce":11119},{"type":"v1","addr":"172.21.15.193:6822","nonce":11119}]},"public_addr":"172.21.15.193:6808/11119","cluster_addr":"172.21.15.193:6815/11119","heartbeat_back_addr":"172.21.15.193:6825/11119","heartbeat_front_addr":"172.21.15.193:6822/11119","state":["exists","up"]},{"osd":1,"uuid":"72d8fe1d-7d45-41c3-9a3a-c83cb3afee51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":11120},{"type":"v1","addr":"172.21.15.193:6803","nonce":11120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":11120},{"type":"v1","addr":"172.21.15.193:6809","nonce":11120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6817","nonce":11120},{"type":"v1","addr":"172.21.15.193:6820","nonce":11120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6811","nonce":11120},{"type":"v1","addr":"172.21.15.193:6814","nonce":11120}]},"public_addr":"172.21.15.193:6803/11120","cluster_addr":"172.21.15.193:6809/11120","heartbeat_back_addr":"172.21.15.193:6820/11120","heartbeat_front_addr":"172.21.15.193:6814/11120","state":["exists","up"]},{"osd":2,"uuid":"7c234f2d-8854-4e13-af2b-8c5d113e1824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":11121},{"type":"v1","addr":"172.21.15.193:6807","nonce":11121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":11121},{"type":"v1","addr":"172.21.15.193:6812","nonce":11121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":11121},{"type":"v1","addr":"172.21.15.193:6823","nonce":11121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":11121},{"type":"v1","addr":"172.21.15.193:6818","nonce":11121}]},"public_addr":"172.21.15.193:6807/11121","cluster_addr":"172.21.15.193:6812/11121","heartbeat_back_addr":"172.21.15.193:6823/11121","heartbeat_front_addr":"172.21.15.193:6818/11121","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:27.040 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:50:27.040 INFO:tasks.ceph:Creating RBD pool 2020-05-24T13:50:27.041 INFO:teuthology.orchestra.run.smithi193:> sudo ceph --cluster c2 osd pool create rbd 8 2020-05-24T13:50:28.682 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'rbd' created 2020-05-24T13:50:28.704 INFO:teuthology.orchestra.run.smithi193:> sudo ceph --cluster c2 osd pool application enable rbd rbd --yes-i-really-mean-it 2020-05-24T13:50:29.643 INFO:teuthology.orchestra.run.smithi193.stderr:enabled application 'rbd' on pool 'rbd' 2020-05-24T13:50:29.656 INFO:tasks.ceph:Starting mds daemons in cluster c2... 2020-05-24T13:50:29.657 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:30.455 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:50:30.456 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":11,"fsid":"be042078-2e26-4e73-a3c1-b7d9ac081979","created":"2020-05-24 13:50:04.563678","modified":"2020-05-24 13:50:29.558605","last_up_change":"2020-05-24 13:50:25.160755","last_in_change":"2020-05-24 13:50:25.160755","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:50:27.966227","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9273eda0-c3ce-4aa8-976b-bd91d636f366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":11119},{"type":"v1","addr":"172.21.15.193:6808","nonce":11119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6813","nonce":11119},{"type":"v1","addr":"172.21.15.193:6815","nonce":11119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":11119},{"type":"v1","addr":"172.21.15.193:6825","nonce":11119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6819","nonce":11119},{"type":"v1","addr":"172.21.15.193:6822","nonce":11119}]},"public_addr":"172.21.15.193:6808/11119","cluster_addr":"172.21.15.193:6815/11119","heartbeat_back_addr":"172.21.15.193:6825/11119","heartbeat_front_addr":"172.21.15.193:6822/11119","state":["exists","up"]},{"osd":1,"uuid":"72d8fe1d-7d45-41c3-9a3a-c83cb3afee51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":11120},{"type":"v1","addr":"172.21.15.193:6803","nonce":11120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":11120},{"type":"v1","addr":"172.21.15.193:6809","nonce":11120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6817","nonce":11120},{"type":"v1","addr":"172.21.15.193:6820","nonce":11120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6811","nonce":11120},{"type":"v1","addr":"172.21.15.193:6814","nonce":11120}]},"public_addr":"172.21.15.193:6803/11120","cluster_addr":"172.21.15.193:6809/11120","heartbeat_back_addr":"172.21.15.193:6820/11120","heartbeat_front_addr":"172.21.15.193:6814/11120","state":["exists","up"]},{"osd":2,"uuid":"7c234f2d-8854-4e13-af2b-8c5d113e1824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":11121},{"type":"v1","addr":"172.21.15.193:6807","nonce":11121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":11121},{"type":"v1","addr":"172.21.15.193:6812","nonce":11121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":11121},{"type":"v1","addr":"172.21.15.193:6823","nonce":11121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":11121},{"type":"v1","addr":"172.21.15.193:6818","nonce":11121}]},"public_addr":"172.21.15.193:6807/11121","cluster_addr":"172.21.15.193:6812/11121","heartbeat_back_addr":"172.21.15.193:6823/11121","heartbeat_front_addr":"172.21.15.193:6818/11121","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:30.472 INFO:tasks.ceph.ceph_manager.c2:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2020-05-24 13:50:27.966227', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-05-24T13:50:30.473 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd pool get rbd pg_num 2020-05-24T13:50:31.325 INFO:teuthology.orchestra.run.smithi193.stdout:pg_num: 8 2020-05-24T13:50:31.340 INFO:tasks.ceph:Waiting until c2 daemons up and pgs clean... 2020-05-24T13:50:31.341 INFO:tasks.ceph.ceph_manager.c2:waiting for mgr available 2020-05-24T13:50:31.341 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 mgr dump --format=json 2020-05-24T13:50:32.215 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:50:32.229 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":10210},{"type":"v1","addr":"172.21.15.193:6801","nonce":10210}]},"active_addr":"172.21.15.193:6801/10210","active_change":"2020-05-24 13:50:08.029626","available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout_min":{"name":"inventory_cache_timeout_min","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}} 2020-05-24T13:50:32.231 INFO:tasks.ceph.ceph_manager.c2:mgr available! 2020-05-24T13:50:32.535 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:33.356 INFO:teuthology.misc.health.smithi193.stdout: 2020-05-24T13:50:33.357 INFO:teuthology.misc.health.smithi193.stdout:{"epoch":13,"fsid":"be042078-2e26-4e73-a3c1-b7d9ac081979","created":"2020-05-24 13:50:04.563678","modified":"2020-05-24 13:50:31.851206","last_up_change":"2020-05-24 13:50:25.160755","last_in_change":"2020-05-24 13:50:25.160755","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:50:27.966227","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9273eda0-c3ce-4aa8-976b-bd91d636f366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":11119},{"type":"v1","addr":"172.21.15.193:6808","nonce":11119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6813","nonce":11119},{"type":"v1","addr":"172.21.15.193:6815","nonce":11119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":11119},{"type":"v1","addr":"172.21.15.193:6825","nonce":11119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6819","nonce":11119},{"type":"v1","addr":"172.21.15.193:6822","nonce":11119}]},"public_addr":"172.21.15.193:6808/11119","cluster_addr":"172.21.15.193:6815/11119","heartbeat_back_addr":"172.21.15.193:6825/11119","heartbeat_front_addr":"172.21.15.193:6822/11119","state":["exists","up"]},{"osd":1,"uuid":"72d8fe1d-7d45-41c3-9a3a-c83cb3afee51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":11120},{"type":"v1","addr":"172.21.15.193:6803","nonce":11120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":11120},{"type":"v1","addr":"172.21.15.193:6809","nonce":11120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6817","nonce":11120},{"type":"v1","addr":"172.21.15.193:6820","nonce":11120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6811","nonce":11120},{"type":"v1","addr":"172.21.15.193:6814","nonce":11120}]},"public_addr":"172.21.15.193:6803/11120","cluster_addr":"172.21.15.193:6809/11120","heartbeat_back_addr":"172.21.15.193:6820/11120","heartbeat_front_addr":"172.21.15.193:6814/11120","state":["exists","up"]},{"osd":2,"uuid":"7c234f2d-8854-4e13-af2b-8c5d113e1824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":11121},{"type":"v1","addr":"172.21.15.193:6807","nonce":11121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":11121},{"type":"v1","addr":"172.21.15.193:6812","nonce":11121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":11121},{"type":"v1","addr":"172.21.15.193:6823","nonce":11121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":11121},{"type":"v1","addr":"172.21.15.193:6818","nonce":11121}]},"public_addr":"172.21.15.193:6807/11121","cluster_addr":"172.21.15.193:6812/11121","heartbeat_back_addr":"172.21.15.193:6823/11121","heartbeat_front_addr":"172.21.15.193:6818/11121","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:33.377 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:50:33.378 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:34.327 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:50:34.328 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":13,"fsid":"be042078-2e26-4e73-a3c1-b7d9ac081979","created":"2020-05-24 13:50:04.563678","modified":"2020-05-24 13:50:31.851206","last_up_change":"2020-05-24 13:50:25.160755","last_in_change":"2020-05-24 13:50:25.160755","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:50:27.966227","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9273eda0-c3ce-4aa8-976b-bd91d636f366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":11119},{"type":"v1","addr":"172.21.15.193:6808","nonce":11119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6813","nonce":11119},{"type":"v1","addr":"172.21.15.193:6815","nonce":11119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":11119},{"type":"v1","addr":"172.21.15.193:6825","nonce":11119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6819","nonce":11119},{"type":"v1","addr":"172.21.15.193:6822","nonce":11119}]},"public_addr":"172.21.15.193:6808/11119","cluster_addr":"172.21.15.193:6815/11119","heartbeat_back_addr":"172.21.15.193:6825/11119","heartbeat_front_addr":"172.21.15.193:6822/11119","state":["exists","up"]},{"osd":1,"uuid":"72d8fe1d-7d45-41c3-9a3a-c83cb3afee51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":11120},{"type":"v1","addr":"172.21.15.193:6803","nonce":11120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":11120},{"type":"v1","addr":"172.21.15.193:6809","nonce":11120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6817","nonce":11120},{"type":"v1","addr":"172.21.15.193:6820","nonce":11120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6811","nonce":11120},{"type":"v1","addr":"172.21.15.193:6814","nonce":11120}]},"public_addr":"172.21.15.193:6803/11120","cluster_addr":"172.21.15.193:6809/11120","heartbeat_back_addr":"172.21.15.193:6820/11120","heartbeat_front_addr":"172.21.15.193:6814/11120","state":["exists","up"]},{"osd":2,"uuid":"7c234f2d-8854-4e13-af2b-8c5d113e1824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":11121},{"type":"v1","addr":"172.21.15.193:6807","nonce":11121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":11121},{"type":"v1","addr":"172.21.15.193:6812","nonce":11121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":11121},{"type":"v1","addr":"172.21.15.193:6823","nonce":11121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":11121},{"type":"v1","addr":"172.21.15.193:6818","nonce":11121}]},"public_addr":"172.21.15.193:6807/11121","cluster_addr":"172.21.15.193:6812/11121","heartbeat_back_addr":"172.21.15.193:6823/11121","heartbeat_front_addr":"172.21.15.193:6818/11121","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:34.354 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 tell osd.0 flush_pg_stats 2020-05-24T13:50:34.914 INFO:teuthology.orchestra.run.smithi193.stdout:38654705667 2020-05-24T13:50:34.934 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 tell osd.1 flush_pg_stats 2020-05-24T13:50:35.578 INFO:teuthology.orchestra.run.smithi193.stdout:34359738372 2020-05-24T13:50:35.594 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 tell osd.2 flush_pg_stats 2020-05-24T13:50:36.148 INFO:teuthology.orchestra.run.smithi193.stdout:34359738372 2020-05-24T13:50:36.160 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd last-stat-seq osd.0 2020-05-24T13:50:37.010 INFO:teuthology.orchestra.run.smithi193.stdout:38654705668 2020-05-24T13:50:37.029 INFO:tasks.ceph.ceph_manager.c2:need seq 38654705667 got 38654705668 for osd.0 2020-05-24T13:50:37.030 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd last-stat-seq osd.1 2020-05-24T13:50:38.042 INFO:teuthology.orchestra.run.smithi193.stdout:34359738372 2020-05-24T13:50:38.060 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738372 got 34359738372 for osd.1 2020-05-24T13:50:38.061 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd last-stat-seq osd.2 2020-05-24T13:50:38.911 INFO:teuthology.orchestra.run.smithi193.stdout:34359738372 2020-05-24T13:50:38.936 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738372 got 34359738372 for osd.2 2020-05-24T13:50:38.936 INFO:tasks.ceph.ceph_manager.c2:waiting for clean 2020-05-24T13:50:38.937 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 pg dump --format=json 2020-05-24T13:50:39.668 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:50:39.669 INFO:teuthology.orchestra.run.smithi193.stderr:dumped all 2020-05-24T13:50:39.686 INFO:teuthology.orchestra.run.smithi193.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2020-05-24 13:50:38.134377","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":87798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.197637","last_change":"2020-05-24 13:50:31.108117","last_active":"2020-05-24 13:50:31.197637","last_peered":"2020-05-24 13:50:31.197637","last_clean":"2020-05-24 13:50:31.197637","last_became_active":"2020-05-24 13:50:31.060314","last_became_peered":"2020-05-24 13:50:31.060314","last_unstale":"2020-05-24 13:50:31.197637","last_undegraded":"2020-05-24 13:50:31.197637","last_fullsized":"2020-05-24 13:50:31.197637","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.216064","last_change":"2020-05-24 13:50:31.180162","last_active":"2020-05-24 13:50:31.216064","last_peered":"2020-05-24 13:50:31.216064","last_clean":"2020-05-24 13:50:31.216064","last_became_active":"2020-05-24 13:50:31.171344","last_became_peered":"2020-05-24 13:50:31.171344","last_unstale":"2020-05-24 13:50:31.216064","last_undegraded":"2020-05-24 13:50:31.216064","last_fullsized":"2020-05-24 13:50:31.216064","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.245477","last_change":"2020-05-24 13:50:31.193438","last_active":"2020-05-24 13:50:31.245477","last_peered":"2020-05-24 13:50:31.245477","last_clean":"2020-05-24 13:50:31.245477","last_became_active":"2020-05-24 13:50:31.152809","last_became_peered":"2020-05-24 13:50:31.152809","last_unstale":"2020-05-24 13:50:31.245477","last_undegraded":"2020-05-24 13:50:31.245477","last_fullsized":"2020-05-24 13:50:31.245477","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.205791","last_change":"2020-05-24 13:50:31.139919","last_active":"2020-05-24 13:50:31.205791","last_peered":"2020-05-24 13:50:31.205791","last_clean":"2020-05-24 13:50:31.205791","last_became_active":"2020-05-24 13:50:31.131121","last_became_peered":"2020-05-24 13:50:31.131121","last_unstale":"2020-05-24 13:50:31.205791","last_undegraded":"2020-05-24 13:50:31.205791","last_fullsized":"2020-05-24 13:50:31.205791","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.230519","last_change":"2020-05-24 13:50:31.222101","last_active":"2020-05-24 13:50:31.230519","last_peered":"2020-05-24 13:50:31.230519","last_clean":"2020-05-24 13:50:31.230519","last_became_active":"2020-05-24 13:50:31.200970","last_became_peered":"2020-05-24 13:50:31.200970","last_unstale":"2020-05-24 13:50:31.230519","last_undegraded":"2020-05-24 13:50:31.230519","last_fullsized":"2020-05-24 13:50:31.230519","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.246901","last_change":"2020-05-24 13:50:31.228474","last_active":"2020-05-24 13:50:31.246901","last_peered":"2020-05-24 13:50:31.246901","last_clean":"2020-05-24 13:50:31.246901","last_became_active":"2020-05-24 13:50:31.160751","last_became_peered":"2020-05-24 13:50:31.160751","last_unstale":"2020-05-24 13:50:31.246901","last_undegraded":"2020-05-24 13:50:31.246901","last_fullsized":"2020-05-24 13:50:31.246901","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.056755","last_change":"2020-05-24 13:50:30.246417","last_active":"2020-05-24 13:50:31.056755","last_peered":"2020-05-24 13:50:31.056755","last_clean":"2020-05-24 13:50:31.056755","last_became_active":"2020-05-24 13:50:30.196976","last_became_peered":"2020-05-24 13:50:30.196976","last_unstale":"2020-05-24 13:50:31.056755","last_undegraded":"2020-05-24 13:50:31.056755","last_fullsized":"2020-05-24 13:50:31.056755","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.195787","last_change":"2020-05-24 13:50:31.102146","last_active":"2020-05-24 13:50:31.195787","last_peered":"2020-05-24 13:50:31.195787","last_clean":"2020-05-24 13:50:31.195787","last_became_active":"2020-05-24 13:50:31.040533","last_became_peered":"2020-05-24 13:50:31.040533","last_unstale":"2020-05-24 13:50:31.195787","last_undegraded":"2020-05-24 13:50:31.195787","last_fullsized":"2020-05-24 13:50:31.195787","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:50:39.688 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 pg dump --format=json 2020-05-24T13:50:40.549 INFO:teuthology.orchestra.run.smithi193.stderr:dumped all 2020-05-24T13:50:40.549 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:50:40.561 INFO:teuthology.orchestra.run.smithi193.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-05-24 13:50:40.134740","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":87798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.197637","last_change":"2020-05-24 13:50:31.108117","last_active":"2020-05-24 13:50:31.197637","last_peered":"2020-05-24 13:50:31.197637","last_clean":"2020-05-24 13:50:31.197637","last_became_active":"2020-05-24 13:50:31.060314","last_became_peered":"2020-05-24 13:50:31.060314","last_unstale":"2020-05-24 13:50:31.197637","last_undegraded":"2020-05-24 13:50:31.197637","last_fullsized":"2020-05-24 13:50:31.197637","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.216064","last_change":"2020-05-24 13:50:31.180162","last_active":"2020-05-24 13:50:31.216064","last_peered":"2020-05-24 13:50:31.216064","last_clean":"2020-05-24 13:50:31.216064","last_became_active":"2020-05-24 13:50:31.171344","last_became_peered":"2020-05-24 13:50:31.171344","last_unstale":"2020-05-24 13:50:31.216064","last_undegraded":"2020-05-24 13:50:31.216064","last_fullsized":"2020-05-24 13:50:31.216064","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.245477","last_change":"2020-05-24 13:50:31.193438","last_active":"2020-05-24 13:50:31.245477","last_peered":"2020-05-24 13:50:31.245477","last_clean":"2020-05-24 13:50:31.245477","last_became_active":"2020-05-24 13:50:31.152809","last_became_peered":"2020-05-24 13:50:31.152809","last_unstale":"2020-05-24 13:50:31.245477","last_undegraded":"2020-05-24 13:50:31.245477","last_fullsized":"2020-05-24 13:50:31.245477","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.205791","last_change":"2020-05-24 13:50:31.139919","last_active":"2020-05-24 13:50:31.205791","last_peered":"2020-05-24 13:50:31.205791","last_clean":"2020-05-24 13:50:31.205791","last_became_active":"2020-05-24 13:50:31.131121","last_became_peered":"2020-05-24 13:50:31.131121","last_unstale":"2020-05-24 13:50:31.205791","last_undegraded":"2020-05-24 13:50:31.205791","last_fullsized":"2020-05-24 13:50:31.205791","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.230519","last_change":"2020-05-24 13:50:31.222101","last_active":"2020-05-24 13:50:31.230519","last_peered":"2020-05-24 13:50:31.230519","last_clean":"2020-05-24 13:50:31.230519","last_became_active":"2020-05-24 13:50:31.200970","last_became_peered":"2020-05-24 13:50:31.200970","last_unstale":"2020-05-24 13:50:31.230519","last_undegraded":"2020-05-24 13:50:31.230519","last_fullsized":"2020-05-24 13:50:31.230519","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.246901","last_change":"2020-05-24 13:50:31.228474","last_active":"2020-05-24 13:50:31.246901","last_peered":"2020-05-24 13:50:31.246901","last_clean":"2020-05-24 13:50:31.246901","last_became_active":"2020-05-24 13:50:31.160751","last_became_peered":"2020-05-24 13:50:31.160751","last_unstale":"2020-05-24 13:50:31.246901","last_undegraded":"2020-05-24 13:50:31.246901","last_fullsized":"2020-05-24 13:50:31.246901","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.056755","last_change":"2020-05-24 13:50:30.246417","last_active":"2020-05-24 13:50:31.056755","last_peered":"2020-05-24 13:50:31.056755","last_clean":"2020-05-24 13:50:31.056755","last_became_active":"2020-05-24 13:50:30.196976","last_became_peered":"2020-05-24 13:50:30.196976","last_unstale":"2020-05-24 13:50:31.056755","last_undegraded":"2020-05-24 13:50:31.056755","last_fullsized":"2020-05-24 13:50:31.056755","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:31.195787","last_change":"2020-05-24 13:50:31.102146","last_active":"2020-05-24 13:50:31.195787","last_peered":"2020-05-24 13:50:31.195787","last_clean":"2020-05-24 13:50:31.195787","last_became_active":"2020-05-24 13:50:31.040533","last_became_peered":"2020-05-24 13:50:31.040533","last_unstale":"2020-05-24 13:50:31.195787","last_undegraded":"2020-05-24 13:50:31.195787","last_fullsized":"2020-05-24 13:50:31.195787","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:28.407109","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:28.407109","last_clean_scrub_stamp":"2020-05-24 13:50:28.407109","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:50:40.563 INFO:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 status --format=json-pretty 2020-05-24T13:50:41.645 INFO:teuthology.orchestra.run.smithi193.stdout: 2020-05-24T13:50:41.646 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2020-05-24T13:50:41.646 INFO:teuthology.orchestra.run.smithi193.stdout: "fsid": "be042078-2e26-4e73-a3c1-b7d9ac081979", 2020-05-24T13:50:41.647 INFO:teuthology.orchestra.run.smithi193.stdout: "health": { 2020-05-24T13:50:41.647 INFO:teuthology.orchestra.run.smithi193.stdout: "checks": {}, 2020-05-24T13:50:41.647 INFO:teuthology.orchestra.run.smithi193.stdout: "status": "HEALTH_OK" 2020-05-24T13:50:41.648 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.648 INFO:teuthology.orchestra.run.smithi193.stdout: "election_epoch": 3, 2020-05-24T13:50:41.648 INFO:teuthology.orchestra.run.smithi193.stdout: "quorum": [ 2020-05-24T13:50:41.648 INFO:teuthology.orchestra.run.smithi193.stdout: 0 2020-05-24T13:50:41.648 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:41.649 INFO:teuthology.orchestra.run.smithi193.stdout: "quorum_names": [ 2020-05-24T13:50:41.649 INFO:teuthology.orchestra.run.smithi193.stdout: "a" 2020-05-24T13:50:41.649 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:41.649 INFO:teuthology.orchestra.run.smithi193.stdout: "quorum_age": 36, 2020-05-24T13:50:41.649 INFO:teuthology.orchestra.run.smithi193.stdout: "monmap": { 2020-05-24T13:50:41.650 INFO:teuthology.orchestra.run.smithi193.stdout: "epoch": 1, 2020-05-24T13:50:41.650 INFO:teuthology.orchestra.run.smithi193.stdout: "fsid": "be042078-2e26-4e73-a3c1-b7d9ac081979", 2020-05-24T13:50:41.650 INFO:teuthology.orchestra.run.smithi193.stdout: "modified": "2020-05-24 13:49:40.775144", 2020-05-24T13:50:41.650 INFO:teuthology.orchestra.run.smithi193.stdout: "created": "2020-05-24 13:49:40.775144", 2020-05-24T13:50:41.650 INFO:teuthology.orchestra.run.smithi193.stdout: "min_mon_release": 14, 2020-05-24T13:50:41.651 INFO:teuthology.orchestra.run.smithi193.stdout: "min_mon_release_name": "nautilus", 2020-05-24T13:50:41.651 INFO:teuthology.orchestra.run.smithi193.stdout: "features": { 2020-05-24T13:50:41.651 INFO:teuthology.orchestra.run.smithi193.stdout: "persistent": [ 2020-05-24T13:50:41.651 INFO:teuthology.orchestra.run.smithi193.stdout: "kraken", 2020-05-24T13:50:41.651 INFO:teuthology.orchestra.run.smithi193.stdout: "luminous", 2020-05-24T13:50:41.652 INFO:teuthology.orchestra.run.smithi193.stdout: "mimic", 2020-05-24T13:50:41.652 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap-prune", 2020-05-24T13:50:41.652 INFO:teuthology.orchestra.run.smithi193.stdout: "nautilus" 2020-05-24T13:50:41.652 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:41.652 INFO:teuthology.orchestra.run.smithi193.stdout: "optional": [] 2020-05-24T13:50:41.653 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.653 INFO:teuthology.orchestra.run.smithi193.stdout: "mons": [ 2020-05-24T13:50:41.653 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.653 INFO:teuthology.orchestra.run.smithi193.stdout: "rank": 0, 2020-05-24T13:50:41.653 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "a", 2020-05-24T13:50:41.654 INFO:teuthology.orchestra.run.smithi193.stdout: "public_addrs": { 2020-05-24T13:50:41.654 INFO:teuthology.orchestra.run.smithi193.stdout: "addrvec": [ 2020-05-24T13:50:41.654 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.654 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v2", 2020-05-24T13:50:41.654 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "172.21.15.193:3300", 2020-05-24T13:50:41.654 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 0 2020-05-24T13:50:41.655 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.655 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.655 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2020-05-24T13:50:41.655 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "172.21.15.193:6789", 2020-05-24T13:50:41.655 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 0 2020-05-24T13:50:41.656 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.656 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2020-05-24T13:50:41.656 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.657 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "172.21.15.193:6789/0", 2020-05-24T13:50:41.657 INFO:teuthology.orchestra.run.smithi193.stdout: "public_addr": "172.21.15.193:6789/0" 2020-05-24T13:50:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2020-05-24T13:50:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap": { 2020-05-24T13:50:41.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap": { 2020-05-24T13:50:41.662 INFO:teuthology.orchestra.run.smithi193.stdout: "epoch": 13, 2020-05-24T13:50:41.662 INFO:teuthology.orchestra.run.smithi193.stdout: "num_osds": 3, 2020-05-24T13:50:41.662 INFO:teuthology.orchestra.run.smithi193.stdout: "num_up_osds": 3, 2020-05-24T13:50:41.663 INFO:teuthology.orchestra.run.smithi193.stdout: "num_in_osds": 3, 2020-05-24T13:50:41.663 INFO:teuthology.orchestra.run.smithi193.stdout: "num_remapped_pgs": 0 2020-05-24T13:50:41.663 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.664 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.664 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap": { 2020-05-24T13:50:41.664 INFO:teuthology.orchestra.run.smithi193.stdout: "pgs_by_state": [ 2020-05-24T13:50:41.664 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.664 INFO:teuthology.orchestra.run.smithi193.stdout: "state_name": "active+clean", 2020-05-24T13:50:41.665 INFO:teuthology.orchestra.run.smithi193.stdout: "count": 8 2020-05-24T13:50:41.665 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.665 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:41.665 INFO:teuthology.orchestra.run.smithi193.stdout: "num_pgs": 8, 2020-05-24T13:50:41.665 INFO:teuthology.orchestra.run.smithi193.stdout: "num_pools": 1, 2020-05-24T13:50:41.665 INFO:teuthology.orchestra.run.smithi193.stdout: "num_objects": 0, 2020-05-24T13:50:41.666 INFO:teuthology.orchestra.run.smithi193.stdout: "data_bytes": 0, 2020-05-24T13:50:41.666 INFO:teuthology.orchestra.run.smithi193.stdout: "bytes_used": 3221594112, 2020-05-24T13:50:41.666 INFO:teuthology.orchestra.run.smithi193.stdout: "bytes_avail": 318900953088, 2020-05-24T13:50:41.667 INFO:teuthology.orchestra.run.smithi193.stdout: "bytes_total": 322122547200 2020-05-24T13:50:41.667 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.667 INFO:teuthology.orchestra.run.smithi193.stdout: "fsmap": { 2020-05-24T13:50:41.668 INFO:teuthology.orchestra.run.smithi193.stdout: "epoch": 1, 2020-05-24T13:50:41.668 INFO:teuthology.orchestra.run.smithi193.stdout: "by_rank": [], 2020-05-24T13:50:41.668 INFO:teuthology.orchestra.run.smithi193.stdout: "up:standby": 0 2020-05-24T13:50:41.669 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.669 INFO:teuthology.orchestra.run.smithi193.stdout: "mgrmap": { 2020-05-24T13:50:41.669 INFO:teuthology.orchestra.run.smithi193.stdout: "epoch": 3, 2020-05-24T13:50:41.669 INFO:teuthology.orchestra.run.smithi193.stdout: "active_gid": 4098, 2020-05-24T13:50:41.669 INFO:teuthology.orchestra.run.smithi193.stdout: "active_name": "x", 2020-05-24T13:50:41.670 INFO:teuthology.orchestra.run.smithi193.stdout: "active_addrs": { 2020-05-24T13:50:41.670 INFO:teuthology.orchestra.run.smithi193.stdout: "addrvec": [ 2020-05-24T13:50:41.670 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.670 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v2", 2020-05-24T13:50:41.670 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "172.21.15.193:6800", 2020-05-24T13:50:41.670 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 10210 2020-05-24T13:50:41.671 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.671 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.671 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2020-05-24T13:50:41.671 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "172.21.15.193:6801", 2020-05-24T13:50:41.671 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 10210 2020-05-24T13:50:41.672 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.672 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2020-05-24T13:50:41.672 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.672 INFO:teuthology.orchestra.run.smithi193.stdout: "active_addr": "172.21.15.193:6801/10210", 2020-05-24T13:50:41.672 INFO:teuthology.orchestra.run.smithi193.stdout: "active_change": "2020-05-24 13:50:08.029626", 2020-05-24T13:50:41.673 INFO:teuthology.orchestra.run.smithi193.stdout: "available": true, 2020-05-24T13:50:41.673 INFO:teuthology.orchestra.run.smithi193.stdout: "standbys": [], 2020-05-24T13:50:41.673 INFO:teuthology.orchestra.run.smithi193.stdout: "modules": [ 2020-05-24T13:50:41.673 INFO:teuthology.orchestra.run.smithi193.stdout: "iostat", 2020-05-24T13:50:41.673 INFO:teuthology.orchestra.run.smithi193.stdout: "restful" 2020-05-24T13:50:41.673 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:41.674 INFO:teuthology.orchestra.run.smithi193.stdout: "available_modules": [ 2020-05-24T13:50:41.674 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.674 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "alerts", 2020-05-24T13:50:41.674 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.675 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.675 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.675 INFO:teuthology.orchestra.run.smithi193.stdout: "interval": { 2020-05-24T13:50:41.675 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "interval", 2020-05-24T13:50:41.675 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.676 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.676 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.676 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "60", 2020-05-24T13:50:41.676 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.676 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.677 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.677 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "How frequently to reexamine health status", 2020-05-24T13:50:41.677 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.678 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.678 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.679 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.679 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_destination": { 2020-05-24T13:50:41.679 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_destination", 2020-05-24T13:50:41.679 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.679 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.680 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.680 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.680 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.680 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.680 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.681 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Email address to send alerts to", 2020-05-24T13:50:41.681 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.682 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.683 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.683 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.683 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_from_name": { 2020-05-24T13:50:41.683 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_from_name", 2020-05-24T13:50:41.683 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.684 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.684 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.684 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "Ceph", 2020-05-24T13:50:41.684 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.684 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.685 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.685 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Email From: name", 2020-05-24T13:50:41.685 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.685 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.686 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.686 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.686 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_host": { 2020-05-24T13:50:41.686 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_host", 2020-05-24T13:50:41.686 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.687 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.687 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.688 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.688 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.690 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.691 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.691 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "SMTP server", 2020-05-24T13:50:41.691 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.691 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.692 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.692 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.692 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_password": { 2020-05-24T13:50:41.692 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_password", 2020-05-24T13:50:41.693 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.693 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.693 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.693 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.693 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.694 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.694 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.694 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Password to authenticate with", 2020-05-24T13:50:41.694 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.694 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.695 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.695 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.695 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_port": { 2020-05-24T13:50:41.695 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_port", 2020-05-24T13:50:41.695 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.696 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.696 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.696 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "465", 2020-05-24T13:50:41.696 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.696 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.697 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.697 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "SMTP port", 2020-05-24T13:50:41.697 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.698 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.698 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.698 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.698 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_sender": { 2020-05-24T13:50:41.699 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_sender", 2020-05-24T13:50:41.699 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.699 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.700 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.700 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.700 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.700 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.701 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.701 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "SMTP envelope sender", 2020-05-24T13:50:41.701 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.701 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.703 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.703 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.704 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_ssl": { 2020-05-24T13:50:41.704 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_ssl", 2020-05-24T13:50:41.704 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.704 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.704 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.705 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.705 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.705 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.705 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.705 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Use SSL to connect to SMTP server", 2020-05-24T13:50:41.706 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.706 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.706 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.706 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.706 INFO:teuthology.orchestra.run.smithi193.stdout: "smtp_user": { 2020-05-24T13:50:41.707 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "smtp_user", 2020-05-24T13:50:41.707 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.707 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.707 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.708 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.708 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.708 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.708 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.709 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "User to authenticate as", 2020-05-24T13:50:41.709 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.709 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.710 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.710 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.711 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.711 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.711 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.711 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ansible", 2020-05-24T13:50:41.711 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.712 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.712 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.712 INFO:teuthology.orchestra.run.smithi193.stdout: "password": { 2020-05-24T13:50:41.712 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "password", 2020-05-24T13:50:41.712 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.713 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.713 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.713 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.713 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.713 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.714 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.714 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.714 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.714 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.715 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.715 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.715 INFO:teuthology.orchestra.run.smithi193.stdout: "server_url": { 2020-05-24T13:50:41.715 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "server_url", 2020-05-24T13:50:41.715 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.716 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.716 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.716 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.716 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.717 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.717 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.717 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.717 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.717 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.717 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.718 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.718 INFO:teuthology.orchestra.run.smithi193.stdout: "username": { 2020-05-24T13:50:41.718 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "username", 2020-05-24T13:50:41.718 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.718 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.719 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.719 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.719 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.719 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.719 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.720 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.720 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.720 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.720 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.720 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.720 INFO:teuthology.orchestra.run.smithi193.stdout: "verify_server": { 2020-05-24T13:50:41.721 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "verify_server", 2020-05-24T13:50:41.721 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.727 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.728 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.728 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.728 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.728 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.728 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.729 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.729 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.729 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.729 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.729 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.730 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.730 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.730 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.730 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "balancer", 2020-05-24T13:50:41.730 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.731 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.731 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.731 INFO:teuthology.orchestra.run.smithi193.stdout: "active": { 2020-05-24T13:50:41.731 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "active", 2020-05-24T13:50:41.731 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.731 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.732 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.732 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:41.732 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.732 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.733 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.733 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "automatically balance PGs across cluster", 2020-05-24T13:50:41.733 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.733 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.733 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.734 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.734 INFO:teuthology.orchestra.run.smithi193.stdout: "begin_time": { 2020-05-24T13:50:41.734 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "begin_time", 2020-05-24T13:50:41.734 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.734 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.735 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.735 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "0000", 2020-05-24T13:50:41.735 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.735 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.735 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.735 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "beginning time of day to automatically balance", 2020-05-24T13:50:41.736 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:50:41.736 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.736 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.736 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.736 INFO:teuthology.orchestra.run.smithi193.stdout: "begin_weekday": { 2020-05-24T13:50:41.737 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "begin_weekday", 2020-05-24T13:50:41.737 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "uint", 2020-05-24T13:50:41.737 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.737 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.737 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "0", 2020-05-24T13:50:41.738 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "0", 2020-05-24T13:50:41.738 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "7", 2020-05-24T13:50:41.738 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.738 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2020-05-24T13:50:41.738 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:50:41.739 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.739 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.739 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.739 INFO:teuthology.orchestra.run.smithi193.stdout: "crush_compat_max_iterations": { 2020-05-24T13:50:41.740 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "crush_compat_max_iterations", 2020-05-24T13:50:41.740 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "uint", 2020-05-24T13:50:41.740 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.740 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.740 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "25", 2020-05-24T13:50:41.741 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "1", 2020-05-24T13:50:41.741 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "250", 2020-05-24T13:50:41.741 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.741 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "maximum number of iterations to attempt optimization", 2020-05-24T13:50:41.741 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.742 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.742 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.742 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.742 INFO:teuthology.orchestra.run.smithi193.stdout: "crush_compat_metrics": { 2020-05-24T13:50:41.742 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "crush_compat_metrics", 2020-05-24T13:50:41.742 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.743 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.743 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.743 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "pgs,objects,bytes", 2020-05-24T13:50:41.743 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.743 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.744 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.744 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "metrics with which to calculate OSD utilization", 2020-05-24T13:50:41.744 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2020-05-24T13:50:41.744 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.744 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.745 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.745 INFO:teuthology.orchestra.run.smithi193.stdout: "crush_compat_step": { 2020-05-24T13:50:41.745 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "crush_compat_step", 2020-05-24T13:50:41.745 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "float", 2020-05-24T13:50:41.745 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.745 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.746 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "0.5", 2020-05-24T13:50:41.746 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "0.001", 2020-05-24T13:50:41.746 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "0.999", 2020-05-24T13:50:41.746 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.746 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "aggressiveness of optimization", 2020-05-24T13:50:41.747 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2020-05-24T13:50:41.747 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.747 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.747 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.747 INFO:teuthology.orchestra.run.smithi193.stdout: "end_time": { 2020-05-24T13:50:41.747 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "end_time", 2020-05-24T13:50:41.748 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.748 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.748 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.748 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "2400", 2020-05-24T13:50:41.748 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.749 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.749 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.749 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "ending time of day to automatically balance", 2020-05-24T13:50:41.749 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:50:41.749 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.749 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.750 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.750 INFO:teuthology.orchestra.run.smithi193.stdout: "end_weekday": { 2020-05-24T13:50:41.750 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "end_weekday", 2020-05-24T13:50:41.750 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "uint", 2020-05-24T13:50:41.750 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.751 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.751 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "7", 2020-05-24T13:50:41.751 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "0", 2020-05-24T13:50:41.751 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "7", 2020-05-24T13:50:41.752 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.752 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2020-05-24T13:50:41.752 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:50:41.752 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.753 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.753 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.753 INFO:teuthology.orchestra.run.smithi193.stdout: "min_score": { 2020-05-24T13:50:41.753 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "min_score", 2020-05-24T13:50:41.754 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "float", 2020-05-24T13:50:41.754 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.754 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.754 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "0", 2020-05-24T13:50:41.754 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.755 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.755 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.755 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "minimum score, below which no optimization is attempted", 2020-05-24T13:50:41.755 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.755 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.755 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.756 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.756 INFO:teuthology.orchestra.run.smithi193.stdout: "mode": { 2020-05-24T13:50:41.756 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "mode", 2020-05-24T13:50:41.756 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.756 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.757 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.757 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "none", 2020-05-24T13:50:41.757 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.758 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.758 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [ 2020-05-24T13:50:41.758 INFO:teuthology.orchestra.run.smithi193.stdout: "crush-compat", 2020-05-24T13:50:41.758 INFO:teuthology.orchestra.run.smithi193.stdout: "none", 2020-05-24T13:50:41.758 INFO:teuthology.orchestra.run.smithi193.stdout: "upmap" 2020-05-24T13:50:41.759 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:41.759 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Balancer mode", 2020-05-24T13:50:41.759 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.759 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.759 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.759 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.760 INFO:teuthology.orchestra.run.smithi193.stdout: "pool_ids": { 2020-05-24T13:50:41.760 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "pool_ids", 2020-05-24T13:50:41.760 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.760 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.760 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.761 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.761 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.761 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.761 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.761 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "pools which the automatic balancing will be limited to", 2020-05-24T13:50:41.761 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.762 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.762 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.762 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.762 INFO:teuthology.orchestra.run.smithi193.stdout: "sleep_interval": { 2020-05-24T13:50:41.762 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "sleep_interval", 2020-05-24T13:50:41.763 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.763 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.763 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.763 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "60", 2020-05-24T13:50:41.763 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.763 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.764 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.764 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "how frequently to wake up and attempt optimization", 2020-05-24T13:50:41.764 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.764 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.764 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.765 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.765 INFO:teuthology.orchestra.run.smithi193.stdout: "upmap_max_deviation": { 2020-05-24T13:50:41.765 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "upmap_max_deviation", 2020-05-24T13:50:41.765 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.766 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.766 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.766 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "5", 2020-05-24T13:50:41.766 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "1", 2020-05-24T13:50:41.766 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.767 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.767 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "deviation below which no optimization is attempted", 2020-05-24T13:50:41.767 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2020-05-24T13:50:41.767 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.767 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.767 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.768 INFO:teuthology.orchestra.run.smithi193.stdout: "upmap_max_iterations": { 2020-05-24T13:50:41.768 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "upmap_max_iterations", 2020-05-24T13:50:41.768 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "uint", 2020-05-24T13:50:41.768 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.768 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.769 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "10", 2020-05-24T13:50:41.769 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.769 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.769 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.769 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "maximum upmap optimization iterations", 2020-05-24T13:50:41.770 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.770 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.770 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.777 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.778 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.778 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.778 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.778 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "crash", 2020-05-24T13:50:41.779 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.779 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.779 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.779 INFO:teuthology.orchestra.run.smithi193.stdout: "retain_interval": { 2020-05-24T13:50:41.780 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "retain_interval", 2020-05-24T13:50:41.780 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.780 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.780 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.780 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "31536000", 2020-05-24T13:50:41.781 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.781 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.781 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.781 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "how long to retain crashes before pruning them", 2020-05-24T13:50:41.781 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.782 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.782 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.782 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.782 INFO:teuthology.orchestra.run.smithi193.stdout: "warn_recent_interval": { 2020-05-24T13:50:41.782 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "warn_recent_interval", 2020-05-24T13:50:41.783 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.783 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.783 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.783 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "1209600", 2020-05-24T13:50:41.783 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.784 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.784 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.784 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "time interval in which to warn about recent crashes", 2020-05-24T13:50:41.784 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.784 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.784 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.785 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.785 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.785 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.785 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.785 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "dashboard", 2020-05-24T13:50:41.786 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.786 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.787 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.787 INFO:teuthology.orchestra.run.smithi193.stdout: "ALERTMANAGER_API_HOST": { 2020-05-24T13:50:41.787 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ALERTMANAGER_API_HOST", 2020-05-24T13:50:41.788 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.788 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.788 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.788 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.788 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.789 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.789 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.789 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.789 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.789 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.790 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.790 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.790 INFO:teuthology.orchestra.run.smithi193.stdout: "AUDIT_API_ENABLED": { 2020-05-24T13:50:41.790 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "AUDIT_API_ENABLED", 2020-05-24T13:50:41.790 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.791 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.791 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.792 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:41.792 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.792 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.792 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.792 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.793 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.793 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.793 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.794 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.794 INFO:teuthology.orchestra.run.smithi193.stdout: "AUDIT_API_LOG_PAYLOAD": { 2020-05-24T13:50:41.794 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2020-05-24T13:50:41.794 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.795 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.795 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.795 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.796 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.796 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.796 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.796 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.797 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.797 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.797 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.798 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.798 INFO:teuthology.orchestra.run.smithi193.stdout: "ENABLE_BROWSABLE_API": { 2020-05-24T13:50:41.798 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ENABLE_BROWSABLE_API", 2020-05-24T13:50:41.798 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.799 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.799 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.799 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.800 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.800 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.800 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.801 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.801 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.801 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.801 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.802 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.802 INFO:teuthology.orchestra.run.smithi193.stdout: "FEATURE_TOGGLE_cephfs": { 2020-05-24T13:50:41.802 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "FEATURE_TOGGLE_cephfs", 2020-05-24T13:50:41.803 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.803 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.803 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.804 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.804 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.804 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.805 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.805 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.805 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.806 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.806 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.806 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.807 INFO:teuthology.orchestra.run.smithi193.stdout: "FEATURE_TOGGLE_iscsi": { 2020-05-24T13:50:41.807 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "FEATURE_TOGGLE_iscsi", 2020-05-24T13:50:41.807 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.808 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.808 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.809 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.809 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.809 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.810 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.810 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.810 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.811 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.811 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.811 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.812 INFO:teuthology.orchestra.run.smithi193.stdout: "FEATURE_TOGGLE_mirroring": { 2020-05-24T13:50:41.812 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "FEATURE_TOGGLE_mirroring", 2020-05-24T13:50:41.812 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.813 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.813 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.813 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.814 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.814 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.814 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.815 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.815 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.816 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.816 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.816 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.817 INFO:teuthology.orchestra.run.smithi193.stdout: "FEATURE_TOGGLE_rbd": { 2020-05-24T13:50:41.817 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "FEATURE_TOGGLE_rbd", 2020-05-24T13:50:41.818 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.818 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.818 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.819 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.819 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.819 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.820 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.820 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.821 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.821 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.821 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.822 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.822 INFO:teuthology.orchestra.run.smithi193.stdout: "FEATURE_TOGGLE_rgw": { 2020-05-24T13:50:41.822 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "FEATURE_TOGGLE_rgw", 2020-05-24T13:50:41.823 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.823 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.823 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.824 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.824 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.824 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.825 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.825 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.825 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.826 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.826 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.826 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.827 INFO:teuthology.orchestra.run.smithi193.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2020-05-24T13:50:41.827 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2020-05-24T13:50:41.827 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.828 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.828 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.828 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.828 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.828 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.828 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.829 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.829 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.829 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.829 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.829 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.830 INFO:teuthology.orchestra.run.smithi193.stdout: "GRAFANA_API_PASSWORD": { 2020-05-24T13:50:41.830 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "GRAFANA_API_PASSWORD", 2020-05-24T13:50:41.830 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.830 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.830 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.831 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "admin", 2020-05-24T13:50:41.831 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.831 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.831 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.831 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.832 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.832 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.832 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.832 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.832 INFO:teuthology.orchestra.run.smithi193.stdout: "GRAFANA_API_SSL_VERIFY": { 2020-05-24T13:50:41.832 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2020-05-24T13:50:41.833 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.833 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.833 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.833 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.833 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.834 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.834 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.835 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.836 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.836 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.836 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.837 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.837 INFO:teuthology.orchestra.run.smithi193.stdout: "GRAFANA_API_URL": { 2020-05-24T13:50:41.837 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "GRAFANA_API_URL", 2020-05-24T13:50:41.838 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.838 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.838 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.839 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.839 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.839 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.840 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.840 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.841 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.841 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.841 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.842 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.842 INFO:teuthology.orchestra.run.smithi193.stdout: "GRAFANA_API_USERNAME": { 2020-05-24T13:50:41.842 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "GRAFANA_API_USERNAME", 2020-05-24T13:50:41.843 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.843 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.843 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.844 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "admin", 2020-05-24T13:50:41.844 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.844 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2020-05-24T13:50:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2020-05-24T13:50:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.848 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.848 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.848 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:41.849 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.849 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.849 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.850 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.850 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.851 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.851 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.851 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.852 INFO:teuthology.orchestra.run.smithi193.stdout: "ISCSI_API_SSL_VERIFICATION": { 2020-05-24T13:50:41.852 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2020-05-24T13:50:41.852 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.853 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.853 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.853 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.854 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.855 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.855 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.855 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.856 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.857 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.857 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.857 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.858 INFO:teuthology.orchestra.run.smithi193.stdout: "PROMETHEUS_API_HOST": { 2020-05-24T13:50:41.858 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "PROMETHEUS_API_HOST", 2020-05-24T13:50:41.858 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.859 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.859 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.859 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.860 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.860 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.860 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.861 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.861 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.861 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.862 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.862 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.862 INFO:teuthology.orchestra.run.smithi193.stdout: "REST_REQUESTS_TIMEOUT": { 2020-05-24T13:50:41.862 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "REST_REQUESTS_TIMEOUT", 2020-05-24T13:50:41.863 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.863 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.863 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.863 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "45", 2020-05-24T13:50:41.863 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.864 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.864 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.864 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.864 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.864 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.865 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.865 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.866 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_ACCESS_KEY": { 2020-05-24T13:50:41.866 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_ACCESS_KEY", 2020-05-24T13:50:41.866 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.867 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.867 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.867 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.867 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.868 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.868 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.868 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.868 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.869 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.869 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.869 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.870 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_ADMIN_RESOURCE": { 2020-05-24T13:50:41.870 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2020-05-24T13:50:41.870 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.870 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.871 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.871 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "admin", 2020-05-24T13:50:41.871 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.871 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.871 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.872 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.872 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.872 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.873 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.873 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.873 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_HOST": { 2020-05-24T13:50:41.874 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_HOST", 2020-05-24T13:50:41.874 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.874 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.874 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.874 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.875 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.875 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.875 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.875 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.876 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.876 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.876 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.877 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.877 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_PORT": { 2020-05-24T13:50:41.877 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_PORT", 2020-05-24T13:50:41.878 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.878 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.879 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.879 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "80", 2020-05-24T13:50:41.879 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.880 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.880 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.880 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.881 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.881 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.881 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.882 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.882 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_SCHEME": { 2020-05-24T13:50:41.882 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_SCHEME", 2020-05-24T13:50:41.883 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.883 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.883 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.883 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "http", 2020-05-24T13:50:41.884 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.884 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.884 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.884 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.885 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.885 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.885 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.885 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.885 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_SECRET_KEY": { 2020-05-24T13:50:41.886 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_SECRET_KEY", 2020-05-24T13:50:41.886 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.886 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.886 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.886 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.887 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.887 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.887 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.887 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.887 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.887 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.888 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.888 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.888 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_SSL_VERIFY": { 2020-05-24T13:50:41.888 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_SSL_VERIFY", 2020-05-24T13:50:41.888 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.889 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.889 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.889 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.889 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.889 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.889 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.890 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.890 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.890 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.890 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.890 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.891 INFO:teuthology.orchestra.run.smithi193.stdout: "RGW_API_USER_ID": { 2020-05-24T13:50:41.891 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "RGW_API_USER_ID", 2020-05-24T13:50:41.891 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.891 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.891 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.892 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.892 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.892 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.892 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.892 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.892 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.893 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.893 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.893 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.893 INFO:teuthology.orchestra.run.smithi193.stdout: "crt_file": { 2020-05-24T13:50:41.893 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "crt_file", 2020-05-24T13:50:41.894 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.894 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.894 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.894 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.894 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.894 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.895 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.895 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.895 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.895 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.895 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.896 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.896 INFO:teuthology.orchestra.run.smithi193.stdout: "debug": { 2020-05-24T13:50:41.896 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "debug", 2020-05-24T13:50:41.897 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.898 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.898 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.898 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:41.898 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.899 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.899 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.899 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Enable/disable debug options", 2020-05-24T13:50:41.899 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.899 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.899 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.900 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.900 INFO:teuthology.orchestra.run.smithi193.stdout: "jwt_token_ttl": { 2020-05-24T13:50:41.900 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "jwt_token_ttl", 2020-05-24T13:50:41.900 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.900 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.901 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.901 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "28800", 2020-05-24T13:50:41.901 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.901 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.901 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.902 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.902 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.902 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.902 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.902 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.903 INFO:teuthology.orchestra.run.smithi193.stdout: "key_file": { 2020-05-24T13:50:41.903 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "key_file", 2020-05-24T13:50:41.903 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.903 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.903 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.904 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.904 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.904 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.904 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.904 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.904 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.905 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.905 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.905 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.905 INFO:teuthology.orchestra.run.smithi193.stdout: "password": { 2020-05-24T13:50:41.905 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "password", 2020-05-24T13:50:41.906 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.906 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.906 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.906 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.906 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.907 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.907 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.907 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.907 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.907 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.908 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.908 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.908 INFO:teuthology.orchestra.run.smithi193.stdout: "server_addr": { 2020-05-24T13:50:41.908 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "server_addr", 2020-05-24T13:50:41.909 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.909 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.909 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.909 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "::", 2020-05-24T13:50:41.910 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.910 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.910 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.910 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.910 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.911 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.911 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.911 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.911 INFO:teuthology.orchestra.run.smithi193.stdout: "server_port": { 2020-05-24T13:50:41.911 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "server_port", 2020-05-24T13:50:41.912 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.912 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.912 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.912 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "8080", 2020-05-24T13:50:41.912 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.913 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.913 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.913 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.913 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.913 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.913 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: "ssl": { 2020-05-24T13:50:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ssl", 2020-05-24T13:50:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.915 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.915 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.915 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.915 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.915 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.915 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.916 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.916 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.916 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.916 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.916 INFO:teuthology.orchestra.run.smithi193.stdout: "ssl_server_port": { 2020-05-24T13:50:41.917 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ssl_server_port", 2020-05-24T13:50:41.917 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.917 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.917 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.917 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "8443", 2020-05-24T13:50:41.917 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.918 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.918 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.918 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.918 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.918 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.919 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.919 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.919 INFO:teuthology.orchestra.run.smithi193.stdout: "standby_behaviour": { 2020-05-24T13:50:41.919 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "standby_behaviour", 2020-05-24T13:50:41.919 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.920 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.920 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.920 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "redirect", 2020-05-24T13:50:41.920 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.920 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.921 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [ 2020-05-24T13:50:41.921 INFO:teuthology.orchestra.run.smithi193.stdout: "error", 2020-05-24T13:50:41.921 INFO:teuthology.orchestra.run.smithi193.stdout: "redirect" 2020-05-24T13:50:41.921 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:41.921 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.921 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.922 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.922 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.922 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.922 INFO:teuthology.orchestra.run.smithi193.stdout: "standby_error_status_code": { 2020-05-24T13:50:41.922 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "standby_error_status_code", 2020-05-24T13:50:41.923 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:41.923 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.923 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.923 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "500", 2020-05-24T13:50:41.923 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "400", 2020-05-24T13:50:41.924 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "599", 2020-05-24T13:50:41.924 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.924 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.924 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.924 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.924 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.925 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.925 INFO:teuthology.orchestra.run.smithi193.stdout: "url_prefix": { 2020-05-24T13:50:41.925 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "url_prefix", 2020-05-24T13:50:41.925 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.925 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.926 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.926 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.926 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.926 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.927 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.927 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.927 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.927 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.927 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.928 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.928 INFO:teuthology.orchestra.run.smithi193.stdout: "username": { 2020-05-24T13:50:41.928 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "username", 2020-05-24T13:50:41.928 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.928 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.929 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.929 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.930 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.930 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.930 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.930 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.931 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.931 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.932 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.932 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.932 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.932 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.932 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.932 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "deepsea", 2020-05-24T13:50:41.933 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.933 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.933 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.933 INFO:teuthology.orchestra.run.smithi193.stdout: "salt_api_eauth": { 2020-05-24T13:50:41.933 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "salt_api_eauth", 2020-05-24T13:50:41.934 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.935 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.935 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.935 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "sharedsecret", 2020-05-24T13:50:41.935 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.936 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.936 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.936 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.936 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.936 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.936 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.937 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.937 INFO:teuthology.orchestra.run.smithi193.stdout: "salt_api_password": { 2020-05-24T13:50:41.937 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "salt_api_password", 2020-05-24T13:50:41.937 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.937 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.938 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.938 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.938 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.938 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.938 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.939 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.939 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.939 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.939 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.939 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.940 INFO:teuthology.orchestra.run.smithi193.stdout: "salt_api_url": { 2020-05-24T13:50:41.940 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "salt_api_url", 2020-05-24T13:50:41.940 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.940 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.940 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.940 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.941 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.941 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.941 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.941 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.941 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.942 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.942 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.942 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.942 INFO:teuthology.orchestra.run.smithi193.stdout: "salt_api_username": { 2020-05-24T13:50:41.942 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "salt_api_username", 2020-05-24T13:50:41.943 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.943 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.943 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.943 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.943 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.943 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.944 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.944 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.944 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.944 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.944 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.945 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.945 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.945 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.945 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.945 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "devicehealth", 2020-05-24T13:50:41.945 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.946 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.946 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.946 INFO:teuthology.orchestra.run.smithi193.stdout: "enable_monitoring": { 2020-05-24T13:50:41.946 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "enable_monitoring", 2020-05-24T13:50:41.946 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.947 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.947 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.947 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:41.947 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.947 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.948 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.948 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "monitor device health metrics", 2020-05-24T13:50:41.948 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.948 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.948 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.948 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.949 INFO:teuthology.orchestra.run.smithi193.stdout: "mark_out_threshold": { 2020-05-24T13:50:41.949 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "mark_out_threshold", 2020-05-24T13:50:41.949 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.949 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.949 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.950 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "2419200", 2020-05-24T13:50:41.950 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.950 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.950 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.950 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "automatically mark OSD if it may fail before this long", 2020-05-24T13:50:41.950 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.951 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.951 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.951 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.951 INFO:teuthology.orchestra.run.smithi193.stdout: "pool_name": { 2020-05-24T13:50:41.951 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "pool_name", 2020-05-24T13:50:41.952 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.952 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.952 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.952 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "device_health_metrics", 2020-05-24T13:50:41.952 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.952 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.953 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.953 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "name of pool in which to store device health metrics", 2020-05-24T13:50:41.953 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.953 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.953 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.954 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.954 INFO:teuthology.orchestra.run.smithi193.stdout: "retention_period": { 2020-05-24T13:50:41.954 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "retention_period", 2020-05-24T13:50:41.954 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.954 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.954 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.955 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "15552000", 2020-05-24T13:50:41.955 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.955 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.955 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.955 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "how long to retain device health metrics", 2020-05-24T13:50:41.956 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.956 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.956 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.956 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.956 INFO:teuthology.orchestra.run.smithi193.stdout: "scrape_frequency": { 2020-05-24T13:50:41.956 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "scrape_frequency", 2020-05-24T13:50:41.957 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.957 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.957 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.957 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "86400", 2020-05-24T13:50:41.957 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.958 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.958 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.958 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "how frequently to scrape device health metrics", 2020-05-24T13:50:41.958 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.958 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.958 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.959 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.959 INFO:teuthology.orchestra.run.smithi193.stdout: "self_heal": { 2020-05-24T13:50:41.959 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "self_heal", 2020-05-24T13:50:41.959 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:41.959 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.959 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.960 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:41.960 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.960 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.960 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.960 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "preemptively heal cluster around devices that may fail", 2020-05-24T13:50:41.961 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.961 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.961 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.961 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.961 INFO:teuthology.orchestra.run.smithi193.stdout: "sleep_interval": { 2020-05-24T13:50:41.961 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "sleep_interval", 2020-05-24T13:50:41.962 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.962 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.962 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.962 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "600", 2020-05-24T13:50:41.962 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.963 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.963 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.963 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "how frequently to wake up and check device health", 2020-05-24T13:50:41.963 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.963 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.964 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.964 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.964 INFO:teuthology.orchestra.run.smithi193.stdout: "warn_threshold": { 2020-05-24T13:50:41.964 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "warn_threshold", 2020-05-24T13:50:41.964 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:41.964 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.965 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:41.965 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "7257600", 2020-05-24T13:50:41.965 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.965 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.965 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.966 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "raise health warning if OSD may fail before this long", 2020-05-24T13:50:41.966 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.966 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.966 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.966 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.967 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:41.967 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.967 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:41.967 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_cloud", 2020-05-24T13:50:41.967 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:41.967 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:41.968 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:41.968 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_cert_context": { 2020-05-24T13:50:41.968 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_cert_context", 2020-05-24T13:50:41.968 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.968 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.969 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.969 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.969 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.969 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.969 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.970 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.970 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.970 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.970 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.970 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.971 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_default_authority": { 2020-05-24T13:50:41.971 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_default_authority", 2020-05-24T13:50:41.971 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.971 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.971 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.971 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "localhost", 2020-05-24T13:50:41.972 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.972 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.972 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.972 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.972 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.973 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.973 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.973 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.973 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_password": { 2020-05-24T13:50:41.973 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_password", 2020-05-24T13:50:41.974 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.974 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.974 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.981 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.981 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.981 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.981 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.982 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.982 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.982 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.983 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.983 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.983 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_port": { 2020-05-24T13:50:41.983 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_port", 2020-05-24T13:50:41.984 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.984 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.984 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.984 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "31400", 2020-05-24T13:50:41.985 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.985 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.985 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.985 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.985 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.986 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.986 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.986 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.986 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_retrieve_prediction_interval": { 2020-05-24T13:50:41.986 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_retrieve_prediction_interval", 2020-05-24T13:50:41.986 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.987 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.987 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.987 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "43200", 2020-05-24T13:50:41.987 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.987 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.988 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.988 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.988 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.988 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.988 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.989 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.989 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_server": { 2020-05-24T13:50:41.989 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_server", 2020-05-24T13:50:41.989 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.989 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.989 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.990 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:41.990 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.990 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.990 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.990 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.991 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.991 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.991 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.991 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.991 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_ssl_target_name_override": { 2020-05-24T13:50:41.992 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_ssl_target_name_override", 2020-05-24T13:50:41.992 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.992 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.992 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.993 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "localhost", 2020-05-24T13:50:41.993 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.993 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.993 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.994 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.994 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.994 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.994 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:41.995 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:41.997 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_upload_metrics_interval": { 2020-05-24T13:50:41.997 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_upload_metrics_interval", 2020-05-24T13:50:41.997 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:41.997 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:41.998 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:41.998 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "600", 2020-05-24T13:50:41.998 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:41.998 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:41.999 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:41.999 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:41.999 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:41.999 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:41.999 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.000 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.000 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_upload_smart_interval": { 2020-05-24T13:50:42.000 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_upload_smart_interval", 2020-05-24T13:50:42.000 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.000 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.001 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.001 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "43200", 2020-05-24T13:50:42.001 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.001 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.001 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.002 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.002 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.002 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.002 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.002 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.003 INFO:teuthology.orchestra.run.smithi193.stdout: "diskprediction_user": { 2020-05-24T13:50:42.003 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_user", 2020-05-24T13:50:42.003 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.003 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.004 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.004 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.004 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.004 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.004 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.004 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.005 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.005 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.005 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.005 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.006 INFO:teuthology.orchestra.run.smithi193.stdout: "sleep_interval": { 2020-05-24T13:50:42.006 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "sleep_interval", 2020-05-24T13:50:42.006 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.006 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.006 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.006 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "600", 2020-05-24T13:50:42.007 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.007 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.007 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.007 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.008 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.008 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.008 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.008 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.008 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.009 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.009 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.009 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "diskprediction_local", 2020-05-24T13:50:42.009 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.009 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.010 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.010 INFO:teuthology.orchestra.run.smithi193.stdout: "predict_interval": { 2020-05-24T13:50:42.010 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "predict_interval", 2020-05-24T13:50:42.010 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.010 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.010 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.011 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "86400", 2020-05-24T13:50:42.011 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.011 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.011 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.011 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.012 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.012 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.012 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.012 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.012 INFO:teuthology.orchestra.run.smithi193.stdout: "sleep_interval": { 2020-05-24T13:50:42.013 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "sleep_interval", 2020-05-24T13:50:42.013 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.013 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.013 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.013 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "600", 2020-05-24T13:50:42.013 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.014 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.014 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.014 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.014 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.014 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.015 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.015 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.015 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.015 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.015 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.016 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "influx", 2020-05-24T13:50:42.016 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": false, 2020-05-24T13:50:42.016 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "influxdb python module not found", 2020-05-24T13:50:42.016 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.016 INFO:teuthology.orchestra.run.smithi193.stdout: "batch_size": { 2020-05-24T13:50:42.017 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "batch_size", 2020-05-24T13:50:42.017 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.017 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.017 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.017 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "5000", 2020-05-24T13:50:42.018 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.018 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.018 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.019 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.019 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.019 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.019 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.019 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.019 INFO:teuthology.orchestra.run.smithi193.stdout: "database": { 2020-05-24T13:50:42.020 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "database", 2020-05-24T13:50:42.020 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.020 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.020 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.020 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "ceph", 2020-05-24T13:50:42.021 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.021 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.021 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.021 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.021 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.022 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.022 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.022 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.022 INFO:teuthology.orchestra.run.smithi193.stdout: "hostname": { 2020-05-24T13:50:42.022 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "hostname", 2020-05-24T13:50:42.022 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.023 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.023 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.023 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.023 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.023 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.024 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.024 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.024 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.024 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.024 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.024 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.025 INFO:teuthology.orchestra.run.smithi193.stdout: "interval": { 2020-05-24T13:50:42.025 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "interval", 2020-05-24T13:50:42.025 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.025 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.025 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.026 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "30", 2020-05-24T13:50:42.026 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.026 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.026 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.027 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.027 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.027 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.027 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.027 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.027 INFO:teuthology.orchestra.run.smithi193.stdout: "password": { 2020-05-24T13:50:42.028 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "password", 2020-05-24T13:50:42.028 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.028 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.028 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.028 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.029 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.029 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.029 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.029 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.029 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.030 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.030 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.030 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.030 INFO:teuthology.orchestra.run.smithi193.stdout: "port": { 2020-05-24T13:50:42.030 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "port", 2020-05-24T13:50:42.031 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.031 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.031 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.031 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "8086", 2020-05-24T13:50:42.031 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.031 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.032 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.032 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.032 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.032 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.032 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.033 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.033 INFO:teuthology.orchestra.run.smithi193.stdout: "ssl": { 2020-05-24T13:50:42.033 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ssl", 2020-05-24T13:50:42.033 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.033 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.034 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.034 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "false", 2020-05-24T13:50:42.034 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.034 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.034 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.034 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.035 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.035 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.035 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.035 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.035 INFO:teuthology.orchestra.run.smithi193.stdout: "threads": { 2020-05-24T13:50:42.036 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "threads", 2020-05-24T13:50:42.036 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.036 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.036 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.036 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "5", 2020-05-24T13:50:42.037 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.037 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.037 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.037 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.037 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.038 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.038 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.038 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.038 INFO:teuthology.orchestra.run.smithi193.stdout: "username": { 2020-05-24T13:50:42.039 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "username", 2020-05-24T13:50:42.040 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.040 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.040 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.041 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.041 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.041 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.041 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.041 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.042 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.042 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.042 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.042 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.042 INFO:teuthology.orchestra.run.smithi193.stdout: "verify_ssl": { 2020-05-24T13:50:42.043 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "verify_ssl", 2020-05-24T13:50:42.043 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.043 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.043 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.043 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "true", 2020-05-24T13:50:42.044 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.044 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.044 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.044 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.044 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.044 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.045 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.045 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.045 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.045 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.046 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.046 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "insights", 2020-05-24T13:50:42.046 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.046 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.046 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": {} 2020-05-24T13:50:42.047 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.047 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.047 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "iostat", 2020-05-24T13:50:42.047 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.047 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.048 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": {} 2020-05-24T13:50:42.048 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.048 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.048 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "localpool", 2020-05-24T13:50:42.048 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.049 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.049 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.049 INFO:teuthology.orchestra.run.smithi193.stdout: "failure_domain": { 2020-05-24T13:50:42.049 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "failure_domain", 2020-05-24T13:50:42.049 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.049 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.050 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.050 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "host", 2020-05-24T13:50:42.050 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.050 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.050 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.051 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "failure domain for any created local pool", 2020-05-24T13:50:42.051 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.051 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.051 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.051 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.052 INFO:teuthology.orchestra.run.smithi193.stdout: "min_size": { 2020-05-24T13:50:42.052 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "min_size", 2020-05-24T13:50:42.052 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.052 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.052 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.052 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.053 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.053 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.053 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.053 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "default min_size for any created local pool", 2020-05-24T13:50:42.053 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.054 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.054 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.054 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.054 INFO:teuthology.orchestra.run.smithi193.stdout: "num_rep": { 2020-05-24T13:50:42.054 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "num_rep", 2020-05-24T13:50:42.054 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.055 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.055 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.055 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "3", 2020-05-24T13:50:42.055 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.055 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.056 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.056 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "default replica count for any created local pool", 2020-05-24T13:50:42.056 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.056 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.056 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.057 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.057 INFO:teuthology.orchestra.run.smithi193.stdout: "pg_num": { 2020-05-24T13:50:42.057 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "pg_num", 2020-05-24T13:50:42.057 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.057 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.057 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.058 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "128", 2020-05-24T13:50:42.058 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.058 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.058 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.058 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "default pg_num for any created local pool", 2020-05-24T13:50:42.059 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.059 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.059 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.059 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.060 INFO:teuthology.orchestra.run.smithi193.stdout: "prefix": { 2020-05-24T13:50:42.060 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "prefix", 2020-05-24T13:50:42.060 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.060 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.061 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.061 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.061 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.061 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.061 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.061 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "name prefix for any created local pool", 2020-05-24T13:50:42.062 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.062 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.062 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.062 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.062 INFO:teuthology.orchestra.run.smithi193.stdout: "subtree": { 2020-05-24T13:50:42.063 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "subtree", 2020-05-24T13:50:42.063 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.063 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.063 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.063 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "rack", 2020-05-24T13:50:42.064 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.064 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.064 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.064 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "CRUSH level for which to create a local pool", 2020-05-24T13:50:42.065 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.065 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.065 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.065 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.065 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.066 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.066 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.066 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "orchestrator_cli", 2020-05-24T13:50:42.066 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.066 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.066 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.067 INFO:teuthology.orchestra.run.smithi193.stdout: "orchestrator": { 2020-05-24T13:50:42.067 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "orchestrator", 2020-05-24T13:50:42.067 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.067 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.067 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.068 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.068 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.068 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.068 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.068 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.068 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.069 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.069 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.069 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.069 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.069 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.070 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.070 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "pg_autoscaler", 2020-05-24T13:50:42.070 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.070 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.070 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.071 INFO:teuthology.orchestra.run.smithi193.stdout: "sleep_interval": { 2020-05-24T13:50:42.071 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "sleep_interval", 2020-05-24T13:50:42.071 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.071 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.071 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.071 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "60", 2020-05-24T13:50:42.072 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.072 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.072 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.072 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.072 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.073 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.073 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.073 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.073 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.073 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.074 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.074 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "progress", 2020-05-24T13:50:42.074 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.074 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.074 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.074 INFO:teuthology.orchestra.run.smithi193.stdout: "max_completed_events": { 2020-05-24T13:50:42.075 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "max_completed_events", 2020-05-24T13:50:42.075 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.075 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.075 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.075 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "50", 2020-05-24T13:50:42.076 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.076 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.076 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.076 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "number of past completed events to remember", 2020-05-24T13:50:42.076 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.077 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.077 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.077 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.077 INFO:teuthology.orchestra.run.smithi193.stdout: "persist_interval": { 2020-05-24T13:50:42.077 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "persist_interval", 2020-05-24T13:50:42.077 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:42.078 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.078 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 1, 2020-05-24T13:50:42.078 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "5", 2020-05-24T13:50:42.078 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.078 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.079 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.079 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "how frequently to persist completed events", 2020-05-24T13:50:42.079 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.079 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.079 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.079 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.080 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.080 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.080 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.080 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "prometheus", 2020-05-24T13:50:42.080 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.081 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.081 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.081 INFO:teuthology.orchestra.run.smithi193.stdout: "rbd_stats_pools": { 2020-05-24T13:50:42.081 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rbd_stats_pools", 2020-05-24T13:50:42.081 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.082 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.082 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.082 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.082 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.083 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.083 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.083 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.083 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.083 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.084 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.084 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.084 INFO:teuthology.orchestra.run.smithi193.stdout: "rbd_stats_pools_refresh_interval": { 2020-05-24T13:50:42.084 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rbd_stats_pools_refresh_interval", 2020-05-24T13:50:42.084 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.084 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.085 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.085 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.085 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.085 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.085 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: "scrape_interval": { 2020-05-24T13:50:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "scrape_interval", 2020-05-24T13:50:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.089 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.089 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.089 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.089 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.089 INFO:teuthology.orchestra.run.smithi193.stdout: "server_addr": { 2020-05-24T13:50:42.090 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "server_addr", 2020-05-24T13:50:42.090 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.090 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.090 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.090 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.090 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.091 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.091 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.091 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.091 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.091 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.092 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.092 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.092 INFO:teuthology.orchestra.run.smithi193.stdout: "server_port": { 2020-05-24T13:50:42.093 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "server_port", 2020-05-24T13:50:42.093 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.093 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.093 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.093 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.093 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.094 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.094 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.094 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.094 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.094 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.095 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.095 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.095 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.095 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.095 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.095 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rbd_support", 2020-05-24T13:50:42.096 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.096 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.096 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": {} 2020-05-24T13:50:42.096 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.096 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.097 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "restful", 2020-05-24T13:50:42.097 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.097 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.097 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.097 INFO:teuthology.orchestra.run.smithi193.stdout: "key_file": { 2020-05-24T13:50:42.098 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "key_file", 2020-05-24T13:50:42.098 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.098 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.099 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.099 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.099 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.099 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.099 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.099 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.100 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.100 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.100 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.100 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.100 INFO:teuthology.orchestra.run.smithi193.stdout: "server_addr": { 2020-05-24T13:50:42.101 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "server_addr", 2020-05-24T13:50:42.101 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.101 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.101 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.101 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.102 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.102 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.102 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.102 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.102 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.102 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.103 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.103 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.103 INFO:teuthology.orchestra.run.smithi193.stdout: "server_port": { 2020-05-24T13:50:42.103 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "server_port", 2020-05-24T13:50:42.103 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.104 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.104 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.104 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.104 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.104 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.104 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.105 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.105 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.105 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.105 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.105 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.106 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.106 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.106 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.106 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rook", 2020-05-24T13:50:42.106 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.106 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.107 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": {} 2020-05-24T13:50:42.107 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.107 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.107 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "selftest", 2020-05-24T13:50:42.107 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.108 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.108 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.108 INFO:teuthology.orchestra.run.smithi193.stdout: "roption1": { 2020-05-24T13:50:42.108 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "roption1", 2020-05-24T13:50:42.108 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.108 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.109 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.109 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.109 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.109 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.109 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.110 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.110 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.110 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.110 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.110 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.110 INFO:teuthology.orchestra.run.smithi193.stdout: "roption2": { 2020-05-24T13:50:42.111 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "roption2", 2020-05-24T13:50:42.111 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.111 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.112 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.112 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "xyz", 2020-05-24T13:50:42.112 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.112 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.113 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.113 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.113 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.113 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.114 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.114 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.114 INFO:teuthology.orchestra.run.smithi193.stdout: "rwoption1": { 2020-05-24T13:50:42.114 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rwoption1", 2020-05-24T13:50:42.114 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.115 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.115 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.115 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.115 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.115 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.115 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.116 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.116 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.116 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.116 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.116 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.117 INFO:teuthology.orchestra.run.smithi193.stdout: "rwoption2": { 2020-05-24T13:50:42.117 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rwoption2", 2020-05-24T13:50:42.117 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.117 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.117 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.117 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.118 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.118 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.118 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.118 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.118 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.119 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.119 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.119 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.119 INFO:teuthology.orchestra.run.smithi193.stdout: "rwoption3": { 2020-05-24T13:50:42.120 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rwoption3", 2020-05-24T13:50:42.120 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "float", 2020-05-24T13:50:42.120 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.120 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.120 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.121 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.121 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.121 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.121 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.121 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.122 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.122 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.122 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.122 INFO:teuthology.orchestra.run.smithi193.stdout: "rwoption4": { 2020-05-24T13:50:42.122 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rwoption4", 2020-05-24T13:50:42.122 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.123 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.123 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.123 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.123 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.123 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.124 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.124 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.124 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.124 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.124 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.124 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.125 INFO:teuthology.orchestra.run.smithi193.stdout: "rwoption5": { 2020-05-24T13:50:42.125 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rwoption5", 2020-05-24T13:50:42.125 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.125 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.125 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.126 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.126 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.126 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.126 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.126 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.126 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.127 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.127 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.127 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.127 INFO:teuthology.orchestra.run.smithi193.stdout: "rwoption6": { 2020-05-24T13:50:42.127 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "rwoption6", 2020-05-24T13:50:42.128 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.128 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.128 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.128 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:42.128 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.128 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.129 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.129 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.129 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.129 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.129 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.130 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.130 INFO:teuthology.orchestra.run.smithi193.stdout: "testkey": { 2020-05-24T13:50:42.130 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "testkey", 2020-05-24T13:50:42.130 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.130 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.130 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.131 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.131 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.131 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.131 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.131 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.132 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.132 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.132 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.132 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.132 INFO:teuthology.orchestra.run.smithi193.stdout: "testlkey": { 2020-05-24T13:50:42.132 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "testlkey", 2020-05-24T13:50:42.133 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.133 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.133 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.134 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.134 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.135 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.135 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.135 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.135 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.135 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.136 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.136 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.136 INFO:teuthology.orchestra.run.smithi193.stdout: "testnewline": { 2020-05-24T13:50:42.137 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "testnewline", 2020-05-24T13:50:42.137 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.137 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.137 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.137 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.138 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.138 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.138 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.138 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.138 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.139 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.139 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.139 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.139 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.139 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.140 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.140 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ssh", 2020-05-24T13:50:42.140 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.140 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.140 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.140 INFO:teuthology.orchestra.run.smithi193.stdout: "inventory_cache_timeout_min": { 2020-05-24T13:50:42.141 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "inventory_cache_timeout_min", 2020-05-24T13:50:42.141 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.141 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.141 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.142 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.142 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.142 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.142 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.142 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.143 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.143 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.143 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.143 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.144 INFO:teuthology.orchestra.run.smithi193.stdout: "ssh_config_file": { 2020-05-24T13:50:42.144 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "ssh_config_file", 2020-05-24T13:50:42.144 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.144 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.144 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.145 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.145 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.145 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.145 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.145 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.145 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.146 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.146 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.146 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.146 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.146 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.147 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.147 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "status", 2020-05-24T13:50:42.147 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.147 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.147 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": {} 2020-05-24T13:50:42.148 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.148 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.148 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "telegraf", 2020-05-24T13:50:42.148 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.148 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.148 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.149 INFO:teuthology.orchestra.run.smithi193.stdout: "address": { 2020-05-24T13:50:42.149 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "address", 2020-05-24T13:50:42.149 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.149 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.149 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.150 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2020-05-24T13:50:42.150 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.150 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.150 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.150 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.151 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.151 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.151 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.151 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.151 INFO:teuthology.orchestra.run.smithi193.stdout: "interval": { 2020-05-24T13:50:42.152 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "interval", 2020-05-24T13:50:42.152 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:42.152 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.152 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.152 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "15", 2020-05-24T13:50:42.153 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.153 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.153 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.153 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.154 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.154 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.154 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.154 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.155 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.155 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.155 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "telemetry", 2020-05-24T13:50:42.156 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.156 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.156 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.156 INFO:teuthology.orchestra.run.smithi193.stdout: "channel_basic": { 2020-05-24T13:50:42.156 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "channel_basic", 2020-05-24T13:50:42.156 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.157 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.157 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.157 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:42.157 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.157 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.158 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.158 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "Share basic cluster information (size, version)", 2020-05-24T13:50:42.158 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.158 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.158 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.159 INFO:teuthology.orchestra.run.smithi193.stdout: "channel_crash": { 2020-05-24T13:50:42.159 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "channel_crash", 2020-05-24T13:50:42.159 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.160 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.160 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.160 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:42.160 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.161 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.161 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.162 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.162 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.162 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.162 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.163 INFO:teuthology.orchestra.run.smithi193.stdout: "channel_device": { 2020-05-24T13:50:42.163 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "channel_device", 2020-05-24T13:50:42.163 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.164 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.164 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.164 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "True", 2020-05-24T13:50:42.164 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.164 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.165 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.165 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.165 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.165 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.165 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.166 INFO:teuthology.orchestra.run.smithi193.stdout: "channel_ident": { 2020-05-24T13:50:42.166 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "channel_ident", 2020-05-24T13:50:42.166 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.166 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.166 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.167 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:42.167 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.167 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.167 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.167 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.168 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.168 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.168 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.168 INFO:teuthology.orchestra.run.smithi193.stdout: "contact": { 2020-05-24T13:50:42.169 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "contact", 2020-05-24T13:50:42.169 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.169 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.169 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.170 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.170 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.170 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.170 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.170 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.171 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.171 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.171 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.171 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.171 INFO:teuthology.orchestra.run.smithi193.stdout: "description": { 2020-05-24T13:50:42.172 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "description", 2020-05-24T13:50:42.172 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.172 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.172 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.172 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.172 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.173 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.173 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.173 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.173 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.173 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.174 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.174 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.174 INFO:teuthology.orchestra.run.smithi193.stdout: "device_url": { 2020-05-24T13:50:42.174 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "device_url", 2020-05-24T13:50:42.174 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.174 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.175 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.175 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "https://telemetry.ceph.com/device", 2020-05-24T13:50:42.175 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.175 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.175 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.176 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.176 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.176 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.176 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.176 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.176 INFO:teuthology.orchestra.run.smithi193.stdout: "enabled": { 2020-05-24T13:50:42.177 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "enabled", 2020-05-24T13:50:42.177 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.177 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.178 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.178 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:42.179 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.179 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.180 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.180 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.180 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.181 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.181 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.181 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.182 INFO:teuthology.orchestra.run.smithi193.stdout: "interval": { 2020-05-24T13:50:42.182 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "interval", 2020-05-24T13:50:42.183 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.183 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.184 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.184 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "24", 2020-05-24T13:50:42.184 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "8", 2020-05-24T13:50:42.185 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.185 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.185 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.186 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.186 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.186 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.187 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.187 INFO:teuthology.orchestra.run.smithi193.stdout: "last_opt_revision": { 2020-05-24T13:50:42.187 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "last_opt_revision", 2020-05-24T13:50:42.188 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.188 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.188 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.189 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "1", 2020-05-24T13:50:42.189 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.189 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.190 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.190 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.190 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.191 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.191 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.191 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.191 INFO:teuthology.orchestra.run.smithi193.stdout: "leaderboard": { 2020-05-24T13:50:42.192 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "leaderboard", 2020-05-24T13:50:42.192 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "bool", 2020-05-24T13:50:42.192 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.193 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.193 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "False", 2020-05-24T13:50:42.193 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.194 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.194 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.194 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.195 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.195 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.196 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.196 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.196 INFO:teuthology.orchestra.run.smithi193.stdout: "organization": { 2020-05-24T13:50:42.197 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "organization", 2020-05-24T13:50:42.197 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.197 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.198 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.198 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.198 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.199 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.199 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.199 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.199 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.199 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.200 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.200 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.200 INFO:teuthology.orchestra.run.smithi193.stdout: "proxy": { 2020-05-24T13:50:42.200 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "proxy", 2020-05-24T13:50:42.200 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.201 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.201 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.201 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.201 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.201 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.202 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.202 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.202 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.203 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.203 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.203 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.203 INFO:teuthology.orchestra.run.smithi193.stdout: "url": { 2020-05-24T13:50:42.203 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "url", 2020-05-24T13:50:42.204 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.204 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.205 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.205 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "https://telemetry.ceph.com/report", 2020-05-24T13:50:42.205 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.206 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.206 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.206 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.206 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.207 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.207 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.207 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.207 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.208 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.208 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.208 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "test_orchestrator", 2020-05-24T13:50:42.208 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.208 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.209 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": {} 2020-05-24T13:50:42.209 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.209 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.209 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "volumes", 2020-05-24T13:50:42.210 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.210 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.210 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": {} 2020-05-24T13:50:42.210 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.210 INFO:teuthology.orchestra.run.smithi193.stdout: { 2020-05-24T13:50:42.210 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "zabbix", 2020-05-24T13:50:42.211 INFO:teuthology.orchestra.run.smithi193.stdout: "can_run": true, 2020-05-24T13:50:42.211 INFO:teuthology.orchestra.run.smithi193.stdout: "error_string": "", 2020-05-24T13:50:42.211 INFO:teuthology.orchestra.run.smithi193.stdout: "module_options": { 2020-05-24T13:50:42.211 INFO:teuthology.orchestra.run.smithi193.stdout: "identifier": { 2020-05-24T13:50:42.211 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "identifier", 2020-05-24T13:50:42.212 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.212 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.212 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.212 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "", 2020-05-24T13:50:42.212 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.213 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.213 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.213 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.213 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.213 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.214 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.214 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.214 INFO:teuthology.orchestra.run.smithi193.stdout: "interval": { 2020-05-24T13:50:42.214 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "interval", 2020-05-24T13:50:42.214 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "secs", 2020-05-24T13:50:42.215 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.215 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.215 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "60", 2020-05-24T13:50:42.215 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.215 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.216 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.216 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.216 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.216 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.216 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.216 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.217 INFO:teuthology.orchestra.run.smithi193.stdout: "zabbix_host": { 2020-05-24T13:50:42.217 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "zabbix_host", 2020-05-24T13:50:42.217 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.217 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.217 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.218 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "None", 2020-05-24T13:50:42.218 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.218 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.218 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.218 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.218 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.219 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.219 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.219 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.219 INFO:teuthology.orchestra.run.smithi193.stdout: "zabbix_port": { 2020-05-24T13:50:42.219 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "zabbix_port", 2020-05-24T13:50:42.220 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "int", 2020-05-24T13:50:42.220 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.220 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.220 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "10051", 2020-05-24T13:50:42.220 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.220 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.221 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.221 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.221 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.221 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.221 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.222 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.222 INFO:teuthology.orchestra.run.smithi193.stdout: "zabbix_sender": { 2020-05-24T13:50:42.222 INFO:teuthology.orchestra.run.smithi193.stdout: "name": "zabbix_sender", 2020-05-24T13:50:42.222 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "str", 2020-05-24T13:50:42.222 INFO:teuthology.orchestra.run.smithi193.stdout: "level": "advanced", 2020-05-24T13:50:42.222 INFO:teuthology.orchestra.run.smithi193.stdout: "flags": 0, 2020-05-24T13:50:42.223 INFO:teuthology.orchestra.run.smithi193.stdout: "default_value": "/usr/bin/zabbix_sender", 2020-05-24T13:50:42.223 INFO:teuthology.orchestra.run.smithi193.stdout: "min": "", 2020-05-24T13:50:42.224 INFO:teuthology.orchestra.run.smithi193.stdout: "max": "", 2020-05-24T13:50:42.224 INFO:teuthology.orchestra.run.smithi193.stdout: "enum_allowed": [], 2020-05-24T13:50:42.225 INFO:teuthology.orchestra.run.smithi193.stdout: "desc": "", 2020-05-24T13:50:42.225 INFO:teuthology.orchestra.run.smithi193.stdout: "long_desc": "", 2020-05-24T13:50:42.226 INFO:teuthology.orchestra.run.smithi193.stdout: "tags": [], 2020-05-24T13:50:42.226 INFO:teuthology.orchestra.run.smithi193.stdout: "see_also": [] 2020-05-24T13:50:42.227 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.227 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.227 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.227 INFO:teuthology.orchestra.run.smithi193.stdout: ], 2020-05-24T13:50:42.228 INFO:teuthology.orchestra.run.smithi193.stdout: "services": {}, 2020-05-24T13:50:42.228 INFO:teuthology.orchestra.run.smithi193.stdout: "always_on_modules": { 2020-05-24T13:50:42.228 INFO:teuthology.orchestra.run.smithi193.stdout: "nautilus": [ 2020-05-24T13:50:42.229 INFO:teuthology.orchestra.run.smithi193.stdout: "balancer", 2020-05-24T13:50:42.229 INFO:teuthology.orchestra.run.smithi193.stdout: "crash", 2020-05-24T13:50:42.229 INFO:teuthology.orchestra.run.smithi193.stdout: "devicehealth", 2020-05-24T13:50:42.230 INFO:teuthology.orchestra.run.smithi193.stdout: "orchestrator_cli", 2020-05-24T13:50:42.230 INFO:teuthology.orchestra.run.smithi193.stdout: "progress", 2020-05-24T13:50:42.231 INFO:teuthology.orchestra.run.smithi193.stdout: "rbd_support", 2020-05-24T13:50:42.231 INFO:teuthology.orchestra.run.smithi193.stdout: "status", 2020-05-24T13:50:42.231 INFO:teuthology.orchestra.run.smithi193.stdout: "volumes" 2020-05-24T13:50:42.231 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2020-05-24T13:50:42.231 INFO:teuthology.orchestra.run.smithi193.stdout: } 2020-05-24T13:50:42.232 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.232 INFO:teuthology.orchestra.run.smithi193.stdout: "servicemap": { 2020-05-24T13:50:42.232 INFO:teuthology.orchestra.run.smithi193.stdout: "epoch": 1, 2020-05-24T13:50:42.232 INFO:teuthology.orchestra.run.smithi193.stdout: "modified": "0.000000", 2020-05-24T13:50:42.232 INFO:teuthology.orchestra.run.smithi193.stdout: "services": {} 2020-05-24T13:50:42.233 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2020-05-24T13:50:42.233 INFO:teuthology.orchestra.run.smithi193.stdout: "progress_events": {} 2020-05-24T13:50:42.233 INFO:teuthology.orchestra.run.smithi193.stdout:} 2020-05-24T13:50:42.235 INFO:tasks.ceph.ceph_manager.c2:{'fsid': 'be042078-2e26-4e73-a3c1-b7d9ac081979', 'health': {'checks': {}, 'status': 'HEALTH_OK'}, 'election_epoch': 3, 'quorum': [0], 'quorum_names': ['a'], 'quorum_age': 36, 'monmap': {'epoch': 1, 'fsid': 'be042078-2e26-4e73-a3c1-b7d9ac081979', 'modified': '2020-05-24 13:49:40.775144', 'created': '2020-05-24 13:49:40.775144', 'min_mon_release': 14, 'min_mon_release_name': 'nautilus', 'features': {'persistent': ['kraken', 'luminous', 'mimic', 'osdmap-prune', 'nautilus'], 'optional': []}, 'mons': [{'rank': 0, 'name': 'a', 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:3300', 'nonce': 0}, {'type': 'v1', 'addr': '172.21.15.193:6789', 'nonce': 0}]}, 'addr': '172.21.15.193:6789/0', 'public_addr': '172.21.15.193:6789/0'}]}, 'osdmap': {'osdmap': {'epoch': 13, 'num_osds': 3, 'num_up_osds': 3, 'num_in_osds': 3, 'num_remapped_pgs': 0}}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 8}], 'num_pgs': 8, 'num_pools': 1, 'num_objects': 0, 'data_bytes': 0, 'bytes_used': 3221594112, 'bytes_avail': 318900953088, 'bytes_total': 322122547200}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'epoch': 3, 'active_gid': 4098, 'active_name': 'x', 'active_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6800', 'nonce': 10210}, {'type': 'v1', 'addr': '172.21.15.193:6801', 'nonce': 10210}]}, 'active_addr': '172.21.15.193:6801/10210', 'active_change': '2020-05-24 13:50:08.029626', 'available': True, 'standbys': [], 'modules': ['iostat', 'restful'], 'available_modules': [{'name': 'alerts', 'can_run': True, 'error_string': '', 'module_options': {'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'How frequently to reexamine health status', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_destination': {'name': 'smtp_destination', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email address to send alerts to', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_from_name': {'name': 'smtp_from_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'Ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email From: name', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_host': {'name': 'smtp_host', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_password': {'name': 'smtp_password', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Password to authenticate with', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_port': {'name': 'smtp_port', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '465', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP port', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_sender': {'name': 'smtp_sender', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP envelope sender', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_ssl': {'name': 'smtp_ssl', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Use SSL to connect to SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_user': {'name': 'smtp_user', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'User to authenticate as', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ansible', 'can_run': True, 'error_string': '', 'module_options': {'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_url': {'name': 'server_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_server': {'name': 'verify_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'balancer', 'can_run': True, 'error_string': '', 'module_options': {'active': {'name': 'active', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically balance PGs across cluster', 'long_desc': '', 'tags': [], 'see_also': []}, 'begin_time': {'name': 'begin_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '0000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'beginning time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'begin_weekday': {'name': 'begin_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to this day of the week or later', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'crush_compat_max_iterations': {'name': 'crush_compat_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '25', 'min': '1', 'max': '250', 'enum_allowed': [], 'desc': 'maximum number of iterations to attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'crush_compat_metrics': {'name': 'crush_compat_metrics', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'pgs,objects,bytes', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'metrics with which to calculate OSD utilization', 'long_desc': 'Value is a list of one or more of "pgs", "objects", or "bytes", and indicates which metrics to use to balance utilization.', 'tags': [], 'see_also': []}, 'crush_compat_step': {'name': 'crush_compat_step', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0.5', 'min': '0.001', 'max': '0.999', 'enum_allowed': [], 'desc': 'aggressiveness of optimization', 'long_desc': '.99 is very aggressive, .01 is less aggressive', 'tags': [], 'see_also': []}, 'end_time': {'name': 'end_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '2400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'ending time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'end_weekday': {'name': 'end_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '7', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to days of the week earlier than this', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'min_score': {'name': 'min_score', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'minimum score, below which no optimization is attempted', 'long_desc': '', 'tags': [], 'see_also': []}, 'mode': {'name': 'mode', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'none', 'min': '', 'max': '', 'enum_allowed': ['crush-compat', 'none', 'upmap'], 'desc': 'Balancer mode', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_ids': {'name': 'pool_ids', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'pools which the automatic balancing will be limited to', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'upmap_max_deviation': {'name': 'upmap_max_deviation', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '1', 'max': '', 'enum_allowed': [], 'desc': 'deviation below which no optimization is attempted', 'long_desc': 'If the number of PGs are within this count then no optimization is attempted', 'tags': [], 'see_also': []}, 'upmap_max_iterations': {'name': 'upmap_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '10', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'maximum upmap optimization iterations', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'crash', 'can_run': True, 'error_string': '', 'module_options': {'retain_interval': {'name': 'retain_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '31536000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain crashes before pruning them', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_recent_interval': {'name': 'warn_recent_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '1209600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'time interval in which to warn about recent crashes', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'dashboard', 'can_run': True, 'error_string': '', 'module_options': {'ALERTMANAGER_API_HOST': {'name': 'ALERTMANAGER_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_ENABLED': {'name': 'AUDIT_API_ENABLED', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_LOG_PAYLOAD': {'name': 'AUDIT_API_LOG_PAYLOAD', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ENABLE_BROWSABLE_API': {'name': 'ENABLE_BROWSABLE_API', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_cephfs': {'name': 'FEATURE_TOGGLE_cephfs', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_iscsi': {'name': 'FEATURE_TOGGLE_iscsi', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_mirroring': {'name': 'FEATURE_TOGGLE_mirroring', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rbd': {'name': 'FEATURE_TOGGLE_rbd', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rgw': {'name': 'FEATURE_TOGGLE_rgw', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE': {'name': 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_PASSWORD': {'name': 'GRAFANA_API_PASSWORD', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_SSL_VERIFY': {'name': 'GRAFANA_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_URL': {'name': 'GRAFANA_API_URL', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_USERNAME': {'name': 'GRAFANA_API_USERNAME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_UPDATE_DASHBOARDS': {'name': 'GRAFANA_UPDATE_DASHBOARDS', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ISCSI_API_SSL_VERIFICATION': {'name': 'ISCSI_API_SSL_VERIFICATION', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'PROMETHEUS_API_HOST': {'name': 'PROMETHEUS_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'REST_REQUESTS_TIMEOUT': {'name': 'REST_REQUESTS_TIMEOUT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '45', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ACCESS_KEY': {'name': 'RGW_API_ACCESS_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ADMIN_RESOURCE': {'name': 'RGW_API_ADMIN_RESOURCE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_HOST': {'name': 'RGW_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_PORT': {'name': 'RGW_API_PORT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '80', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SCHEME': {'name': 'RGW_API_SCHEME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'http', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SECRET_KEY': {'name': 'RGW_API_SECRET_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SSL_VERIFY': {'name': 'RGW_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_USER_ID': {'name': 'RGW_API_USER_ID', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'crt_file': {'name': 'crt_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'debug': {'name': 'debug', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Enable/disable debug options', 'long_desc': '', 'tags': [], 'see_also': []}, 'jwt_token_ttl': {'name': 'jwt_token_ttl', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '28800', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '::', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8080', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl_server_port': {'name': 'ssl_server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8443', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_behaviour': {'name': 'standby_behaviour', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'redirect', 'min': '', 'max': '', 'enum_allowed': ['error', 'redirect'], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_error_status_code': {'name': 'standby_error_status_code', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '500', 'min': '400', 'max': '599', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url_prefix': {'name': 'url_prefix', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'deepsea', 'can_run': True, 'error_string': '', 'module_options': {'salt_api_eauth': {'name': 'salt_api_eauth', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'sharedsecret', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_password': {'name': 'salt_api_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_url': {'name': 'salt_api_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_username': {'name': 'salt_api_username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'devicehealth', 'can_run': True, 'error_string': '', 'module_options': {'enable_monitoring': {'name': 'enable_monitoring', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'monitor device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'mark_out_threshold': {'name': 'mark_out_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '2419200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically mark OSD if it may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_name': {'name': 'pool_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'device_health_metrics', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name of pool in which to store device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'retention_period': {'name': 'retention_period', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '15552000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_frequency': {'name': 'scrape_frequency', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to scrape device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'self_heal': {'name': 'self_heal', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'preemptively heal cluster around devices that may fail', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and check device health', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_threshold': {'name': 'warn_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '7257600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'raise health warning if OSD may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_cloud', 'can_run': True, 'error_string': '', 'module_options': {'diskprediction_cert_context': {'name': 'diskprediction_cert_context', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_default_authority': {'name': 'diskprediction_default_authority', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_password': {'name': 'diskprediction_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_port': {'name': 'diskprediction_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '31400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_retrieve_prediction_interval': {'name': 'diskprediction_retrieve_prediction_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_server': {'name': 'diskprediction_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_ssl_target_name_override': {'name': 'diskprediction_ssl_target_name_override', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_metrics_interval': {'name': 'diskprediction_upload_metrics_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_smart_interval': {'name': 'diskprediction_upload_smart_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_user': {'name': 'diskprediction_user', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_local', 'can_run': True, 'error_string': '', 'module_options': {'predict_interval': {'name': 'predict_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'influx', 'can_run': False, 'error_string': 'influxdb python module not found', 'module_options': {'batch_size': {'name': 'batch_size', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'database': {'name': 'database', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'hostname': {'name': 'hostname', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '30', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'port': {'name': 'port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '8086', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'false', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'threads': {'name': 'threads', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_ssl': {'name': 'verify_ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'true', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'insights', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'iostat', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'localpool', 'can_run': True, 'error_string': '', 'module_options': {'failure_domain': {'name': 'failure_domain', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'host', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'failure domain for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'min_size': {'name': 'min_size', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default min_size for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'num_rep': {'name': 'num_rep', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '3', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default replica count for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'pg_num': {'name': 'pg_num', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '128', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default pg_num for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'prefix': {'name': 'prefix', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name prefix for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'subtree': {'name': 'subtree', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'rack', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'CRUSH level for which to create a local pool', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'orchestrator_cli', 'can_run': True, 'error_string': '', 'module_options': {'orchestrator': {'name': 'orchestrator', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'pg_autoscaler', 'can_run': True, 'error_string': '', 'module_options': {'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'progress', 'can_run': True, 'error_string': '', 'module_options': {'max_completed_events': {'name': 'max_completed_events', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '50', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'number of past completed events to remember', 'long_desc': '', 'tags': [], 'see_also': []}, 'persist_interval': {'name': 'persist_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to persist completed events', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'prometheus', 'can_run': True, 'error_string': '', 'module_options': {'rbd_stats_pools': {'name': 'rbd_stats_pools', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rbd_stats_pools_refresh_interval': {'name': 'rbd_stats_pools_refresh_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_interval': {'name': 'scrape_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rbd_support', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'restful', 'can_run': True, 'error_string': '', 'module_options': {'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rook', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'selftest', 'can_run': True, 'error_string': '', 'module_options': {'roption1': {'name': 'roption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'roption2': {'name': 'roption2', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'xyz', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption1': {'name': 'rwoption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption2': {'name': 'rwoption2', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption3': {'name': 'rwoption3', 'type': 'float', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption4': {'name': 'rwoption4', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption5': {'name': 'rwoption5', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption6': {'name': 'rwoption6', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testkey': {'name': 'testkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testlkey': {'name': 'testlkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testnewline': {'name': 'testnewline', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ssh', 'can_run': True, 'error_string': '', 'module_options': {'inventory_cache_timeout_min': {'name': 'inventory_cache_timeout_min', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssh_config_file': {'name': 'ssh_config_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'status', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'telegraf', 'can_run': True, 'error_string': '', 'module_options': {'address': {'name': 'address', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'unixgram:///tmp/telegraf.sock', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '15', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'telemetry', 'can_run': True, 'error_string': '', 'module_options': {'channel_basic': {'name': 'channel_basic', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Share basic cluster information (size, version)', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_crash': {'name': 'channel_crash', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_device': {'name': 'channel_device', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_ident': {'name': 'channel_ident', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'contact': {'name': 'contact', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'description': {'name': 'description', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'device_url': {'name': 'device_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/device', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '24', 'min': '8', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'last_opt_revision': {'name': 'last_opt_revision', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '1', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'leaderboard': {'name': 'leaderboard', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'organization': {'name': 'organization', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'proxy': {'name': 'proxy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url': {'name': 'url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/report', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'test_orchestrator', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'volumes', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'zabbix', 'can_run': True, 'error_string': '', 'module_options': {'identifier': {'name': 'identifier', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_host': {'name': 'zabbix_host', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_port': {'name': 'zabbix_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '10051', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_sender': {'name': 'zabbix_sender', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '/usr/bin/zabbix_sender', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}], 'services': {}, 'always_on_modules': {'nautilus': ['balancer', 'crash', 'devicehealth', 'orchestrator_cli', 'progress', 'rbd_support', 'status', 'volumes']}}, 'servicemap': {'epoch': 1, 'modified': '0.000000', 'services': {}}, 'progress_events': {}} 2020-05-24T13:50:42.238 INFO:tasks.ceph.ceph_manager.c2:clean! 2020-05-24T13:50:42.238 INFO:tasks.ceph:Waiting until ceph cluster c2 is healthy... 2020-05-24T13:50:42.239 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 health 2020-05-24T13:50:43.059 INFO:teuthology.misc.health.smithi193.stdout:HEALTH_OK 2020-05-24T13:50:43.071 DEBUG:teuthology.misc:Ceph health: HEALTH_OK 2020-05-24T13:50:43.071 INFO:teuthology.run_tasks:Running task rgw... 2020-05-24T13:50:43.088 DEBUG:tasks.rgw:config is {'c1.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c1.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2020-05-24T13:50:43.089 DEBUG:tasks.rgw:client list is dict_keys(['c1.client.0', 'c1.client.1', 'c2.client.0', 'c2.client.1']) 2020-05-24T13:50:43.089 INFO:tasks.rgw:Creating data pools 2020-05-24T13:50:43.090 DEBUG:tasks.rgw:Obtaining remote for client c1.client.0 2020-05-24T13:50:43.090 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:50:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'default.rgw.buckets.data' created 2020-05-24T13:50:44.226 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2020-05-24T13:50:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:45.374 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2020-05-24T13:50:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'default.rgw.buckets.index' created 2020-05-24T13:50:46.420 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2020-05-24T13:50:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:50:47.489 DEBUG:tasks.rgw:Obtaining remote for client c1.client.1 2020-05-24T13:50:47.489 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:50:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'default.rgw.buckets.data' already exists 2020-05-24T13:50:48.261 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2020-05-24T13:50:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:49.571 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2020-05-24T13:50:50.333 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'default.rgw.buckets.index' already exists 2020-05-24T13:50:50.358 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2020-05-24T13:50:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:50:51.758 DEBUG:tasks.rgw:Obtaining remote for client c2.client.0 2020-05-24T13:50:51.759 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:50:53.595 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'default.rgw.buckets.data' created 2020-05-24T13:50:53.611 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2020-05-24T13:50:54.588 INFO:teuthology.orchestra.run.smithi193.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:54.606 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2020-05-24T13:50:55.649 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'default.rgw.buckets.index' created 2020-05-24T13:50:55.667 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2020-05-24T13:50:56.724 INFO:teuthology.orchestra.run.smithi193.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:50:56.738 DEBUG:tasks.rgw:Obtaining remote for client c2.client.1 2020-05-24T13:50:56.738 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:50:57.485 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'default.rgw.buckets.data' already exists 2020-05-24T13:50:57.499 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2020-05-24T13:50:58.842 INFO:teuthology.orchestra.run.smithi193.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:58.857 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2020-05-24T13:50:59.568 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'default.rgw.buckets.index' already exists 2020-05-24T13:50:59.581 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2020-05-24T13:51:01.212 INFO:teuthology.orchestra.run.smithi193.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:51:01.227 DEBUG:tasks.rgw:Pools created 2020-05-24T13:51:01.228 INFO:tasks.rgw:Configuring compression type = random 2020-05-24T13:51:01.228 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'list'] 2020-05-24T13:51:01.228 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'list'] 2020-05-24T13:51:01.229 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user list 2020-05-24T13:51:01.549 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:01.547 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:04.960 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:04.960 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:05.073 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:05.073 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:05.098 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:05.097 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:05.098 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:05.097 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:05.287 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.719 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.736 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.735 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:07.736 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.735 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.749 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.748 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:07.749 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.748 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.799 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.799 7f25df257840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:51:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.799 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.823 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.822 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:07.824 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.822 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.843 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.841 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:07.843 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.841 7f25df257840 10 Creating default zonegroup 2020-05-24T13:51:07.843 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.841 7f25df257840 20 get_system_obj_state: rctx=0x7ffc2e85c870 obj=:.pools.avail state=0x55e4a2c2f1b0 s->prefetch_data=0 2020-05-24T13:51:07.845 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:07.844 7f25df257840 10 couldn't find old data placement pools config, setting up new ones for the zone 2020-05-24T13:51:08.451 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:08.450 7f25df257840 10 WARNING: store->list_zones() returned r=-2 2020-05-24T13:51:08.452 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:08.450 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:08.467 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:08.466 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.518 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.554 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.552 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:09.554 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.552 7f25df257840 10 could not read realm id: (2) No such file or directory 2020-05-24T13:51:09.554 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.552 7f25df257840 10 WARNING: failed to set zone as default, r=-22 2020-05-24T13:51:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.552 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.572 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:09.807 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.805 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.873 7f25df257840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.873 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.902 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.901 7f25df257840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:09.902 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.901 7f25df257840 20 zonegroup default 2020-05-24T13:51:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.901 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.915 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.914 7f25df257840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:09.916 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.914 7f25df257840 10 Cannot find current period zone using local zone 2020-05-24T13:51:09.916 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.914 7f25df257840 10 Using default name default 2020-05-24T13:51:09.916 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.914 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.956 7f25df257840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.956 7f25df257840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.982 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.981 7f25df257840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:09.983 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:09.981 7f25df257840 20 zone default 2020-05-24T13:51:14.156 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.155 7f25df257840 20 add_watcher() i=0 2020-05-24T13:51:14.157 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.155 7f25df257840 20 add_watcher() i=1 2020-05-24T13:51:14.157 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.155 7f25df257840 20 add_watcher() i=2 2020-05-24T13:51:14.157 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.155 7f25df257840 20 add_watcher() i=3 2020-05-24T13:51:14.157 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.155 7f25df257840 20 add_watcher() i=4 2020-05-24T13:51:14.158 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.155 7f25df257840 20 add_watcher() i=5 2020-05-24T13:51:14.158 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.155 7f25df257840 20 add_watcher() i=6 2020-05-24T13:51:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.244 7f25df257840 20 add_watcher() i=7 2020-05-24T13:51:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.244 7f25df257840 2 all 8 watchers are set, enabling cache 2020-05-24T13:51:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:14.272 7f25a8ff9700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:15.464 7f25df257840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.meta 2020-05-24T13:51:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:15.467 7f25df257840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.meta 2020-05-24T13:51:17.587 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:17.586 7f25df257840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.log 2020-05-24T13:51:17.590 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:17.589 7f25df257840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.log 2020-05-24T13:51:17.591 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:17.590 7f2582fe5700 20 reqs_thread_entry: start 2020-05-24T13:51:17.592 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:17.590 7f2580fe1700 20 reqs_thread_entry: start 2020-05-24T13:51:17.592 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:17.590 7f25df257840 20 init_complete bucket index max shards: 0 2020-05-24T13:51:17.592 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:17.591 7f251effd700 20 reqs_thread_entry: start 2020-05-24T13:51:23.017 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.015 7f25df257840 20 remove_watcher() i=0 2020-05-24T13:51:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.015 7f25df257840 2 removed watcher, disabling cache 2020-05-24T13:51:23.146 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.144 7f25df257840 20 remove_watcher() i=1 2020-05-24T13:51:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.248 7f25df257840 20 remove_watcher() i=2 2020-05-24T13:51:23.361 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.360 7f25df257840 20 remove_watcher() i=3 2020-05-24T13:51:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.444 7f25df257840 20 remove_watcher() i=4 2020-05-24T13:51:23.527 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.525 7f25df257840 20 remove_watcher() i=5 2020-05-24T13:51:23.604 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.603 7f25df257840 20 remove_watcher() i=6 2020-05-24T13:51:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.714 7f25df257840 20 remove_watcher() i=7 2020-05-24T13:51:23.723 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T13:51:23.723 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:23.724 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:23.724 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:23.725 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:23.934 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.932 7fdccbfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:23.934 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.932 7fdccb7fe700 20 reqs_thread_entry: start 2020-05-24T13:51:23.934 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.932 7fdd06b24840 20 rados->read ofs=0 len=0 2020-05-24T13:51:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.962 7fdd06b24840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.962 7fdd06b24840 20 rados->read ofs=0 len=0 2020-05-24T13:51:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.993 7fdd06b24840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:23.994 7fdd06b24840 20 rados->read ofs=0 len=0 2020-05-24T13:51:24.007 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.005 7fdd06b24840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:24.007 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.005 7fdd06b24840 20 rados->read ofs=0 len=0 2020-05-24T13:51:24.021 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.019 7fdd06b24840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:24.022 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.019 7fdd06b24840 20 rados->read ofs=0 len=0 2020-05-24T13:51:24.035 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.033 7fdd06b24840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:24.144 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"8d50db50-f391-4a39-8539-9c28e6ef2d6c","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:24.144 DEBUG:tasks.util.rgw: json result: {'id': '8d50db50-f391-4a39-8539-9c28e6ef2d6c', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:24.145 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['user', 'list'] 2020-05-24T13:51:24.145 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'user', 'list'] 2020-05-24T13:51:24.146 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 user list 2020-05-24T13:51:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.318 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.350 7f3a3c4b4840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.351 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:24.384 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.382 7f3a3c4b4840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:24.384 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.382 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.392 7f3a3c4b4840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:24.394 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.392 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:24.408 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:24.406 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:25.071 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.070 7f3a3c4b4840 20 RGWRados::pool_iterate: got zonegroup_info.0720c8e2-fbe5-48fa-a780-1aa7de7244d0 2020-05-24T13:51:25.072 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.070 7f3a3c4b4840 20 RGWRados::pool_iterate: got zone_info.8d50db50-f391-4a39-8539-9c28e6ef2d6c 2020-05-24T13:51:25.072 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.070 7f3a3c4b4840 20 RGWRados::pool_iterate: got zone_names.default 2020-05-24T13:51:25.072 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.070 7f3a3c4b4840 20 RGWRados::pool_iterate: got zonegroups_names.default 2020-05-24T13:51:25.072 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.070 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.082 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.082 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.095 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.095 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.110 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.110 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.122 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.121 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:25.122 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.121 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.134 7f3a3c4b4840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.134 7f3a3c4b4840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:51:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.134 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.145 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.144 7f3a3c4b4840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.144 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.166 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.166 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.190 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.188 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:25.190 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.188 7f3a3c4b4840 20 zonegroup default 2020-05-24T13:51:25.190 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.188 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.200 7f3a3c4b4840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.200 7f3a3c4b4840 10 Cannot find current period zone using local zone 2020-05-24T13:51:25.202 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.200 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.212 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.211 7f3a3c4b4840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:25.212 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.211 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.234 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.233 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:25.235 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.233 7f3a3c4b4840 20 rados->read ofs=0 len=0 2020-05-24T13:51:25.256 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.255 7f3a3c4b4840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:25.256 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.255 7f3a3c4b4840 20 zone default 2020-05-24T13:51:25.940 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.938 7f3a3c4b4840 20 add_watcher() i=0 2020-05-24T13:51:25.940 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.938 7f3a3c4b4840 20 add_watcher() i=1 2020-05-24T13:51:25.940 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.938 7f3a3c4b4840 20 add_watcher() i=2 2020-05-24T13:51:25.941 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.938 7f3a3c4b4840 20 add_watcher() i=3 2020-05-24T13:51:25.941 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.938 7f3a3c4b4840 20 add_watcher() i=4 2020-05-24T13:51:25.941 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.938 7f3a3c4b4840 20 add_watcher() i=5 2020-05-24T13:51:25.941 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.938 7f3a3c4b4840 20 add_watcher() i=6 2020-05-24T13:51:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.993 7f3a3c4b4840 20 add_watcher() i=7 2020-05-24T13:51:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:25.993 7f3a3c4b4840 2 all 8 watchers are set, enabling cache 2020-05-24T13:51:26.024 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.023 7f3a05ff3700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:26.032 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.031 7f39cbfff700 20 reqs_thread_entry: start 2020-05-24T13:51:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.031 7f39c9ffb700 20 reqs_thread_entry: start 2020-05-24T13:51:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.031 7f3a3c4b4840 20 init_complete bucket index max shards: 0 2020-05-24T13:51:26.033 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.032 7f3953fff700 20 reqs_thread_entry: start 2020-05-24T13:51:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.706 7f3a3c4b4840 20 remove_watcher() i=0 2020-05-24T13:51:26.708 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.706 7f3a3c4b4840 2 removed watcher, disabling cache 2020-05-24T13:51:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.784 7f3a3c4b4840 20 remove_watcher() i=1 2020-05-24T13:51:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.871 7f3a3c4b4840 20 remove_watcher() i=2 2020-05-24T13:51:26.957 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:26.955 7f3a3c4b4840 20 remove_watcher() i=3 2020-05-24T13:51:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.029 7f3a3c4b4840 20 remove_watcher() i=4 2020-05-24T13:51:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.109 7f3a3c4b4840 20 remove_watcher() i=5 2020-05-24T13:51:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.177 7f3a3c4b4840 20 remove_watcher() i=6 2020-05-24T13:51:27.231 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.229 7f3a3c4b4840 20 remove_watcher() i=7 2020-05-24T13:51:27.236 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T13:51:27.236 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:27.237 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:27.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:27.238 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:27.458 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.456 7f8384ff9700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:27.458 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.456 7f8377fff700 20 reqs_thread_entry: start 2020-05-24T13:51:27.459 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.456 7f83bb398840 20 rados->read ofs=0 len=0 2020-05-24T13:51:27.491 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.489 7f83bb398840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:27.491 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.489 7f83bb398840 20 rados->read ofs=0 len=0 2020-05-24T13:51:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.523 7f83bb398840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.523 7f83bb398840 20 rados->read ofs=0 len=0 2020-05-24T13:51:27.535 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.533 7f83bb398840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:27.535 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.533 7f83bb398840 20 rados->read ofs=0 len=0 2020-05-24T13:51:27.553 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.551 7f83bb398840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:27.553 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.551 7f83bb398840 20 rados->read ofs=0 len=0 2020-05-24T13:51:27.567 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:51:27.565 7f83bb398840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:27.662 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"8d50db50-f391-4a39-8539-9c28e6ef2d6c","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:27.663 DEBUG:tasks.util.rgw: json result: {'id': '8d50db50-f391-4a39-8539-9c28e6ef2d6c', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:27.664 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['user', 'list'] 2020-05-24T13:51:27.664 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'user', 'list'] 2020-05-24T13:51:27.664 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 user list 2020-05-24T13:51:27.961 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:27.962 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:30.306 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:30.307 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:30.306 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:30.307 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:30.731 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:30.732 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:30.731 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:30.732 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:30.741 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:30.742 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:30.741 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:30.742 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:30.766 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:30.768 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:32.434 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.436 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.451 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.452 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:32.452 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.452 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.464 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.465 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:32.465 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.465 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.515 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.516 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:32.515 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.516 7fa308f8c840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:51:32.515 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.516 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.530 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.531 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:32.530 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.531 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.546 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.546 7fa308f8c840 10 Creating default zonegroup 2020-05-24T13:51:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.546 7fa308f8c840 20 get_system_obj_state: rctx=0x7ffe69b13e60 obj=:.pools.avail state=0x563a13a1c1a0 s->prefetch_data=0 2020-05-24T13:51:32.549 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:32.550 7fa308f8c840 10 couldn't find old data placement pools config, setting up new ones for the zone 2020-05-24T13:51:33.179 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:33.180 7fa308f8c840 10 WARNING: store->list_zones() returned r=-2 2020-05-24T13:51:33.180 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:33.181 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:33.195 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:33.196 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:33.998 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.000 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:34.024 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.025 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:34.024 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.025 7fa308f8c840 10 could not read realm id: (2) No such file or directory 2020-05-24T13:51:34.025 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.025 7fa308f8c840 10 WARNING: failed to set zone as default, r=-22 2020-05-24T13:51:34.025 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.025 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:34.053 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.051 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:34.266 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.263 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:34.322 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.323 7fa308f8c840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:34.322 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.323 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:34.374 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.375 7fa308f8c840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:34.374 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.375 7fa308f8c840 20 zonegroup default 2020-05-24T13:51:34.375 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.375 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:34.392 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.393 7fa308f8c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:34.393 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.393 7fa308f8c840 10 Cannot find current period zone using local zone 2020-05-24T13:51:34.393 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.393 7fa308f8c840 10 Using default name default 2020-05-24T13:51:34.393 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.393 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:34.415 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.416 7fa308f8c840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:34.416 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.416 7fa308f8c840 20 rados->read ofs=0 len=0 2020-05-24T13:51:34.438 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.435 7fa308f8c840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:34.438 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:34.435 7fa308f8c840 20 zone default 2020-05-24T13:51:40.123 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.124 7fa308f8c840 20 add_watcher() i=0 2020-05-24T13:51:40.123 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.124 7fa308f8c840 20 add_watcher() i=1 2020-05-24T13:51:40.124 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.124 7fa308f8c840 20 add_watcher() i=2 2020-05-24T13:51:40.124 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.124 7fa308f8c840 20 add_watcher() i=3 2020-05-24T13:51:40.124 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.124 7fa308f8c840 20 add_watcher() i=4 2020-05-24T13:51:40.124 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.124 7fa308f8c840 20 add_watcher() i=5 2020-05-24T13:51:40.124 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.124 7fa308f8c840 20 add_watcher() i=6 2020-05-24T13:51:40.285 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.287 7fa308f8c840 20 add_watcher() i=7 2020-05-24T13:51:40.286 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.287 7fa308f8c840 2 all 8 watchers are set, enabling cache 2020-05-24T13:51:40.383 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:40.384 7fa2c5ffb700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:42.384 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:42.385 7fa308f8c840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.meta 2020-05-24T13:51:42.386 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:42.388 7fa308f8c840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.meta 2020-05-24T13:51:44.497 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:44.497 7fa308f8c840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.log 2020-05-24T13:51:44.499 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:44.501 7fa308f8c840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.log 2020-05-24T13:51:44.501 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:44.502 7fa2a77fe700 20 reqs_thread_entry: start 2020-05-24T13:51:44.501 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:44.502 7fa2a57fa700 20 reqs_thread_entry: start 2020-05-24T13:51:44.501 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:44.502 7fa308f8c840 20 init_complete bucket index max shards: 0 2020-05-24T13:51:44.502 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:44.503 7fa2177fe700 20 reqs_thread_entry: start 2020-05-24T13:51:49.141 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.142 7fa308f8c840 20 remove_watcher() i=0 2020-05-24T13:51:49.141 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.142 7fa308f8c840 2 removed watcher, disabling cache 2020-05-24T13:51:49.198 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.200 7fa308f8c840 20 remove_watcher() i=1 2020-05-24T13:51:49.263 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.265 7fa308f8c840 20 remove_watcher() i=2 2020-05-24T13:51:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.350 7fa308f8c840 20 remove_watcher() i=3 2020-05-24T13:51:49.424 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.425 7fa308f8c840 20 remove_watcher() i=4 2020-05-24T13:51:49.505 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.507 7fa308f8c840 20 remove_watcher() i=5 2020-05-24T13:51:49.592 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.593 7fa308f8c840 20 remove_watcher() i=6 2020-05-24T13:51:49.712 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:49.713 7fa308f8c840 20 remove_watcher() i=7 2020-05-24T13:51:49.720 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T13:51:49.720 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:49.721 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:49.721 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:49.722 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:50.019 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.020 7fe3c0ff9700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:50.019 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.020 7fe3b3fff700 20 reqs_thread_entry: start 2020-05-24T13:51:50.019 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.020 7fe3f742e840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.050 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.052 7fe3f742e840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:50.051 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.052 7fe3f742e840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.086 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.087 7fe3f742e840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:50.087 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.088 7fe3f742e840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.097 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.098 7fe3f742e840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:50.097 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.098 7fe3f742e840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.111 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.113 7fe3f742e840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:50.111 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.113 7fe3f742e840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.124 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.126 7fe3f742e840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:50.268 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"4c2c263a-a02c-4170-afbd-9699f119c090","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:50.268 DEBUG:tasks.util.rgw: json result: {'id': '4c2c263a-a02c-4170-afbd-9699f119c090', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:50.269 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['user', 'list'] 2020-05-24T13:51:50.269 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'user', 'list'] 2020-05-24T13:51:50.270 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 user list 2020-05-24T13:51:50.469 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.471 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.523 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.525 7fabd9a95840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:50.524 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.525 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.580 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.581 7fabd9a95840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:50.580 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.581 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.594 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.595 7fabd9a95840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:50.594 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.595 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:50.611 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:50.613 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:51.222 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.223 7fabd9a95840 20 RGWRados::pool_iterate: got zonegroup_info.dd65ae9b-38f7-4e3e-b8e2-308d755c2957 2020-05-24T13:51:51.222 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.223 7fabd9a95840 20 RGWRados::pool_iterate: got zone_info.4c2c263a-a02c-4170-afbd-9699f119c090 2020-05-24T13:51:51.222 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.223 7fabd9a95840 20 RGWRados::pool_iterate: got zone_names.default 2020-05-24T13:51:51.223 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.223 7fabd9a95840 20 RGWRados::pool_iterate: got zonegroups_names.default 2020-05-24T13:51:51.223 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.223 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.248 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.249 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:51.248 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.249 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.276 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.277 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:51.276 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.278 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.298 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.300 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:51.298 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.300 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.317 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.318 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:51.317 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.318 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.332 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.334 7fabd9a95840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:51.333 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.334 7fabd9a95840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:51:51.333 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.334 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.346 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.348 7fabd9a95840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:51.347 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.348 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.368 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.370 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:51.368 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.370 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.388 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.389 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:51.388 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.389 7fabd9a95840 20 zonegroup default 2020-05-24T13:51:51.388 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.389 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.401 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.402 7fabd9a95840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:51.401 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.402 7fabd9a95840 10 Cannot find current period zone using local zone 2020-05-24T13:51:51.401 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.402 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.413 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.414 7fabd9a95840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:51.413 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.414 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.431 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.433 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:51.431 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.433 7fabd9a95840 20 rados->read ofs=0 len=0 2020-05-24T13:51:51.448 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.449 7fabd9a95840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:51.448 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:51.449 7fabd9a95840 20 zone default 2020-05-24T13:51:52.004 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.005 7fabd9a95840 20 add_watcher() i=0 2020-05-24T13:51:52.004 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.005 7fabd9a95840 20 add_watcher() i=1 2020-05-24T13:51:52.004 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.005 7fabd9a95840 20 add_watcher() i=2 2020-05-24T13:51:52.004 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.005 7fabd9a95840 20 add_watcher() i=3 2020-05-24T13:51:52.004 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.005 7fabd9a95840 20 add_watcher() i=4 2020-05-24T13:51:52.005 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.005 7fabd9a95840 20 add_watcher() i=5 2020-05-24T13:51:52.009 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.011 7fabd9a95840 20 add_watcher() i=6 2020-05-24T13:51:52.055 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.056 7fabd9a95840 20 add_watcher() i=7 2020-05-24T13:51:52.055 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.056 7fabd9a95840 2 all 8 watchers are set, enabling cache 2020-05-24T13:51:52.088 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.089 7fab9effd700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:52.096 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.097 7fab857e2700 20 reqs_thread_entry: start 2020-05-24T13:51:52.096 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.098 7fab4affd700 20 reqs_thread_entry: start 2020-05-24T13:51:52.097 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.098 7fabd9a95840 20 init_complete bucket index max shards: 0 2020-05-24T13:51:52.097 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.098 7fab497fa700 20 reqs_thread_entry: start 2020-05-24T13:51:52.823 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.824 7fabd9a95840 20 remove_watcher() i=0 2020-05-24T13:51:52.823 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.824 7fabd9a95840 2 removed watcher, disabling cache 2020-05-24T13:51:52.885 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.886 7fabd9a95840 20 remove_watcher() i=1 2020-05-24T13:51:52.940 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.942 7fabd9a95840 20 remove_watcher() i=2 2020-05-24T13:51:52.993 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:52.995 7fabd9a95840 20 remove_watcher() i=3 2020-05-24T13:51:53.048 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.049 7fabd9a95840 20 remove_watcher() i=4 2020-05-24T13:51:53.102 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.101 7fabd9a95840 20 remove_watcher() i=5 2020-05-24T13:51:53.159 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.156 7fabd9a95840 20 remove_watcher() i=6 2020-05-24T13:51:53.209 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.210 7fabd9a95840 20 remove_watcher() i=7 2020-05-24T13:51:53.214 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T13:51:53.215 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:53.215 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:53.215 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:53.216 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:53.414 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.415 7f2686ffd700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:53.414 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.415 7f26867fc700 20 reqs_thread_entry: start 2020-05-24T13:51:53.415 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.416 7f26c1cbc840 20 rados->read ofs=0 len=0 2020-05-24T13:51:53.444 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.445 7f26c1cbc840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:53.444 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.445 7f26c1cbc840 20 rados->read ofs=0 len=0 2020-05-24T13:51:53.482 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.483 7f26c1cbc840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:53.482 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.483 7f26c1cbc840 20 rados->read ofs=0 len=0 2020-05-24T13:51:53.494 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.496 7f26c1cbc840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:53.495 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.496 7f26c1cbc840 20 rados->read ofs=0 len=0 2020-05-24T13:51:53.511 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.512 7f26c1cbc840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:53.511 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.512 7f26c1cbc840 20 rados->read ofs=0 len=0 2020-05-24T13:51:53.530 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:51:53.532 7f26c1cbc840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:53.628 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"4c2c263a-a02c-4170-afbd-9699f119c090","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:53.629 DEBUG:tasks.util.rgw: json result: {'id': '4c2c263a-a02c-4170-afbd-9699f119c090', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:53.630 INFO:tasks.rgw:Starting rgw... 2020-05-24T13:51:53.630 INFO:tasks.rgw:rgw c1.client.0 config is {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:53.631 INFO:tasks.rgw:Using beast as radosgw frontend 2020-05-24T13:51:53.632 DEBUG:teuthology.misc:running c1.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:53.632 INFO:tasks.rgw.c1.client.0:Restarting daemon 2020-05-24T13:51:53.634 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2020-05-24T13:51:53.638 INFO:tasks.rgw.c1.client.0:Started 2020-05-24T13:51:53.638 INFO:tasks.rgw:rgw c1.client.1 config is {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:53.639 INFO:tasks.rgw:Using beast as radosgw frontend 2020-05-24T13:51:53.639 DEBUG:teuthology.misc:running c1.client.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.client.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:53.639 INFO:tasks.rgw.c1.client.1:Restarting daemon 2020-05-24T13:51:53.640 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2020-05-24T13:51:53.643 INFO:tasks.rgw.c1.client.1:Started 2020-05-24T13:51:53.643 INFO:tasks.rgw:rgw c2.client.0 config is {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:53.644 INFO:tasks.rgw:Using beast as radosgw frontend 2020-05-24T13:51:53.644 DEBUG:teuthology.misc:running c2.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:53.645 INFO:tasks.rgw.c2.client.0:Restarting daemon 2020-05-24T13:51:53.645 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2020-05-24T13:51:53.648 INFO:tasks.rgw.c2.client.0:Started 2020-05-24T13:51:53.648 INFO:tasks.rgw:rgw c2.client.1 config is {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:53.649 INFO:tasks.rgw:Using beast as radosgw frontend 2020-05-24T13:51:53.649 DEBUG:teuthology.misc:running c2.client.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.client.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:53.649 INFO:tasks.rgw.c2.client.1:Restarting daemon 2020-05-24T13:51:53.650 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2020-05-24T13:51:53.653 INFO:tasks.rgw.c2.client.1:Started 2020-05-24T13:51:53.653 INFO:tasks.rgw:Polling c1.client.0 until it starts accepting connections on http://smithi019.front.sepia.ceph.com:8000/ 2020-05-24T13:51:53.653 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000/ 2020-05-24T13:51:53.752 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:53.752 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:53.756 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:51:53.758 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:51:54.759 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000/ 2020-05-24T13:51:54.823 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:54.823 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:54.825 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:51:54.825 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:51:56.826 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000/ 2020-05-24T13:51:56.903 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:56.904 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:56.912 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:51:56.912 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:52:00.913 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000/ 2020-05-24T13:52:00.969 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:00.970 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:00.970 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:52:00.971 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:52:08.972 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000/ 2020-05-24T13:52:09.024 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:09.025 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:09.407 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 559 0 --:--:-- --:--:-- --:--:-- 560 2020-05-24T13:52:09.408 INFO:teuthology.orchestra.run.smithi019.stdout:anonymous 2020-05-24T13:52:09.408 INFO:tasks.rgw:Polling c1.client.1 until it starts accepting connections on http://smithi019.front.sepia.ceph.com:8001/ 2020-05-24T13:52:09.409 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001/ 2020-05-24T13:52:09.458 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:09.458 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:09.857 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 536 0 --:--:-- --:--:-- --:--:-- 535 2020-05-24T13:52:09.858 INFO:teuthology.orchestra.run.smithi019.stdout:anonymous 2020-05-24T13:52:09.858 INFO:tasks.rgw:Polling c2.client.0 until it starts accepting connections on http://smithi193.front.sepia.ceph.com:8000/ 2020-05-24T13:52:09.859 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8000/ 2020-05-24T13:52:09.999 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:09.999 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:10.293 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 722 0 --:--:-- --:--:-- --:--:-- 722 2020-05-24T13:52:10.293 INFO:teuthology.orchestra.run.smithi193.stdout:anonymous 2020-05-24T13:52:10.294 INFO:tasks.rgw:Polling c2.client.1 until it starts accepting connections on http://smithi193.front.sepia.ceph.com:8001/ 2020-05-24T13:52:10.294 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8001/ 2020-05-24T13:52:10.346 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:10.347 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:10.633 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 747 0 --:--:-- --:--:-- --:--:-- 748 2020-05-24T13:52:10.634 INFO:teuthology.orchestra.run.smithi193.stdout:anonymous 2020-05-24T13:52:10.634 INFO:teuthology.run_tasks:Running task rgw-multisite... 2020-05-24T13:52:10.857 INFO:tasks.rgw_multisite:creating realm.. 2020-05-24T13:52:10.858 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:10.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:10.859 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 realm create --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:12.268 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","name":"test-realm","current_period":"9f6e589d-e881-43af-8021-1a7b7a957375","epoch":1} 2020-05-24T13:52:12.269 INFO:tasks.rgw_multisite:creating master zonegroup.. 2020-05-24T13:52:12.270 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:12.270 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:12.271 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup set --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:12.570 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:52:12.569 7f8f33066840 -1 old format 2020-05-24T13:52:13.109 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[],"default_placement":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T13:52:13.110 INFO:tasks.rgw_multisite:creating master zone.. 2020-05-24T13:52:13.111 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi019.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:13.111 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi019.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:13.112 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --master --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --endpoints http://smithi019.front.sepia.ceph.com:8000 --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:15.977 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T13:52:15.978 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool create test-zone1.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:52:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'test-zone1.rgw.buckets.data' created 2020-05-24T13:52:16.769 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool application enable test-zone1.rgw.buckets.data rgw --cluster c1 2020-05-24T13:52:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rgw' on pool 'test-zone1.rgw.buckets.data' 2020-05-24T13:52:17.868 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:17.868 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:17.869 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:21.582 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:52:21.582 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:21.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:21.583 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:48.995 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","epoch":1,"predecessor_uuid":"9f6e589d-e881-43af-8021-1a7b7a957375","sync_status":[],"period_map":{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"false","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:52:48.997 DEBUG:tasks.rgw.c1.client.0:waiting for process to exit 2020-05-24T13:52:48.997 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:52:49.014 INFO:tasks.rgw.c1.client.0.smithi019.stdout:2020-05-24 13:52:49.001 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground (PID: 12715) UID: 0 2020-05-24T13:52:49.019 INFO:tasks.rgw.c1.client.0.smithi019.stdout:2020-05-24 13:52:49.004 4049040 -1 shutting down 2020-05-24T13:53:01.201 INFO:tasks.rgw.c1.client.0:Stopped 2020-05-24T13:53:01.202 INFO:tasks.rgw.c1.client.0:Restarting daemon 2020-05-24T13:53:01.202 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2020-05-24T13:53:01.206 INFO:tasks.rgw.c1.client.0:Started 2020-05-24T13:53:01.207 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000 2020-05-24T13:53:01.301 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:01.302 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:01.304 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:01.306 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:02.307 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000 2020-05-24T13:53:02.392 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:02.394 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:02.396 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:02.397 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:04.398 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000 2020-05-24T13:53:04.489 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:04.489 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:04.489 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:04.490 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:08.491 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000 2020-05-24T13:53:08.554 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:08.554 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:08.558 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:08.558 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:16.560 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8000 2020-05-24T13:53:16.620 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:16.621 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:17.000 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 560 0 --:--:-- --:--:-- --:--:-- 561 2020-05-24T13:53:17.001 INFO:teuthology.orchestra.run.smithi019.stdout:anonymous 2020-05-24T13:53:17.001 INFO:tasks.rgw_multisite:creating admin user.. 2020-05-24T13:53:17.002 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:17.003 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:17.003 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid realm-admin --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Realm Admin' --system --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:20.160 INFO:teuthology.orchestra.run.smithi019.stdout:{"user_id":"realm-admin","display_name":"Realm Admin","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"realm-admin","access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","system":"true","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2020-05-24T13:53:21.143 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2020-05-24T13:53:21.144 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi019.front.sepia.ceph.com:8000', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:21.144 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi019.front.sepia.ceph.com:8000', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:21.144 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi019.front.sepia.ceph.com:8000 --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:22.968 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","name":"test-realm","current_period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","epoch":2} 2020-05-24T13:53:22.969 INFO:tasks.rgw_multisite:creating zone.. 2020-05-24T13:53:22.970 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi193.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:22.970 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi193.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:22.971 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --endpoints http://smithi193.front.sepia.ceph.com:8000 --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:24.056 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:53:24.057 7fc41c394840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T13:53:24.056 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:53:24.057 7fc41c394840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T13:53:24.201 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T13:53:24.202 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool create test-zone2.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:53:25.465 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'test-zone2.rgw.buckets.data' created 2020-05-24T13:53:25.481 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool application enable test-zone2.rgw.buckets.data rgw --cluster c2 2020-05-24T13:53:26.414 INFO:teuthology.orchestra.run.smithi193.stderr:enabled application 'rgw' on pool 'test-zone2.rgw.buckets.data' 2020-05-24T13:53:26.429 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:26.430 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:26.431 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:31.404 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:53:31.405 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:31.405 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:31.406 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:51.910 INFO:teuthology.orchestra.run.smithi193.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T13:53:55.767 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","epoch":2,"predecessor_uuid":"9f6e589d-e881-43af-8021-1a7b7a957375","sync_status":[],"period_map":{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:53:55.768 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2020-05-24T13:53:55.768 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:53:55.787 INFO:tasks.rgw.c2.client.0.smithi193.stdout:2020-05-24 13:53:55.775 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground (PID: 13498) UID: 0 2020-05-24T13:53:55.789 INFO:tasks.rgw.c2.client.0.smithi193.stdout:2020-05-24 13:53:55.777 4049040 -1 shutting down 2020-05-24T13:54:01.872 INFO:tasks.rgw.c2.client.0:Stopped 2020-05-24T13:54:01.873 INFO:tasks.rgw.c2.client.0:Restarting daemon 2020-05-24T13:54:01.873 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2020-05-24T13:54:01.878 INFO:tasks.rgw.c2.client.0:Started 2020-05-24T13:54:01.879 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8000 2020-05-24T13:54:01.977 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:01.977 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:01.981 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:54:01.983 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:54:02.984 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8000 2020-05-24T13:54:03.069 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:03.069 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:03.074 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:54:03.074 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:54:05.076 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8000 2020-05-24T13:54:05.163 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:05.163 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:05.166 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:54:05.167 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:54:09.168 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8000 2020-05-24T13:54:09.236 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:09.237 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:09.240 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:54:09.240 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:54:17.246 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8000 2020-05-24T13:54:17.298 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:17.298 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:17.626 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 652 0 --:--:-- --:--:-- --:--:-- 654 2020-05-24T13:54:17.626 INFO:teuthology.orchestra.run.smithi193.stdout:anonymous 2020-05-24T13:54:17.627 INFO:tasks.rgw_multisite:creating zone.. 2020-05-24T13:54:17.628 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi019.front.sepia.ceph.com:8001', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:17.628 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi019.front.sepia.ceph.com:8001', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:17.629 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --endpoints http://smithi019.front.sepia.ceph.com:8001 --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:54:18.959 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:54:18.959 7f3410656840 0 failed reading obj info from .rgw.root:zone_info.692d187d-2dfb-4f37-a732-96d22188c425: (2) No such file or directory 2020-05-24T13:54:18.959 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 13:54:18.959 7f3410656840 0 WARNING: could not read zone params for zone id=692d187d-2dfb-4f37-a732-96d22188c425 name=test-zone2 2020-05-24T13:54:19.058 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","domain_root":"test-zone3.rgw.meta:root","control_pool":"test-zone3.rgw.control","gc_pool":"test-zone3.rgw.log:gc","lc_pool":"test-zone3.rgw.log:lc","log_pool":"test-zone3.rgw.log","intent_log_pool":"test-zone3.rgw.log:intent","usage_log_pool":"test-zone3.rgw.log:usage","reshard_pool":"test-zone3.rgw.log:reshard","user_keys_pool":"test-zone3.rgw.meta:users.keys","user_email_pool":"test-zone3.rgw.meta:users.email","user_swift_pool":"test-zone3.rgw.meta:users.swift","user_uid_pool":"test-zone3.rgw.meta:users.uid","otp_pool":"test-zone3.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone3.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone3.rgw.buckets.data"}},"data_extra_pool":"test-zone3.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T13:54:19.059 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool create test-zone3.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:54:20.274 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'test-zone3.rgw.buckets.data' created 2020-05-24T13:54:20.288 INFO:teuthology.orchestra.run.smithi019:> sudo ceph osd pool application enable test-zone3.rgw.buckets.data rgw --cluster c1 2020-05-24T13:54:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:enabled application 'rgw' on pool 'test-zone3.rgw.buckets.data' 2020-05-24T13:54:21.292 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:21.293 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:21.293 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:54:28.493 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","domain_root":"test-zone3.rgw.meta:root","control_pool":"test-zone3.rgw.control","gc_pool":"test-zone3.rgw.log:gc","lc_pool":"test-zone3.rgw.log:lc","log_pool":"test-zone3.rgw.log","intent_log_pool":"test-zone3.rgw.log:intent","usage_log_pool":"test-zone3.rgw.log:usage","reshard_pool":"test-zone3.rgw.log:reshard","user_keys_pool":"test-zone3.rgw.meta:users.keys","user_email_pool":"test-zone3.rgw.meta:users.email","user_swift_pool":"test-zone3.rgw.meta:users.swift","user_uid_pool":"test-zone3.rgw.meta:users.uid","otp_pool":"test-zone3.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone3.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone3.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone3.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:54:28.495 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:28.496 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:28.496 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:54:58.049 INFO:teuthology.orchestra.run.smithi019.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T13:55:01.783 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","epoch":3,"predecessor_uuid":"9f6e589d-e881-43af-8021-1a7b7a957375","sync_status":[],"period_map":{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:55:01.786 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2020-05-24T13:55:01.786 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:55:01.805 INFO:tasks.rgw.c1.client.1.smithi019.stdout:2020-05-24 13:55:01.789 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground (PID: 12719) UID: 0 2020-05-24T13:55:01.809 INFO:tasks.rgw.c1.client.1.smithi019.stdout:2020-05-24 13:55:01.791 4049040 -1 shutting down 2020-05-24T13:55:07.893 INFO:tasks.rgw.c1.client.1:Stopped 2020-05-24T13:55:07.894 INFO:tasks.rgw.c1.client.1:Restarting daemon 2020-05-24T13:55:07.895 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2020-05-24T13:55:07.900 INFO:tasks.rgw.c1.client.1:Started 2020-05-24T13:55:07.900 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T13:55:07.946 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:07.947 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:07.951 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:07.952 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:08.954 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T13:55:09.008 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:09.008 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:09.019 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:09.019 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:11.021 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T13:55:11.074 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:11.075 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:11.076 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:11.077 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:15.079 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T13:55:15.123 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:15.123 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:15.126 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:15.127 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:23.129 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T13:55:23.175 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:23.176 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:23.491 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 691 0 --:--:-- --:--:-- --:--:-- 692 2020-05-24T13:55:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:anonymous 2020-05-24T13:55:23.492 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2020-05-24T13:55:23.492 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi019.front.sepia.ceph.com:8000', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--url', 'http://smithi019.front.sepia.ceph.com:8000', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:23.493 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi019.front.sepia.ceph.com:8000', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--url', 'http://smithi019.front.sepia.ceph.com:8000', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:23.493 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi019.front.sepia.ceph.com:8000 --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --url http://smithi019.front.sepia.ceph.com:8000 --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:55:24.511 INFO:tasks.rgw.c2.client.0.smithi193.stdout:2020-05-24 13:55:24.506 148888700 -1 rgw realm watcher: RGWRealmWatcher::handle_error oid=realms.4aca72c0-4e47-41a6-ae5c-598b4dc05858.control err=-107 2020-05-24T13:55:25.106 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","name":"test-realm","current_period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","epoch":2} 2020-05-24T13:55:25.106 INFO:tasks.rgw_multisite:creating zone.. 2020-05-24T13:55:25.107 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi193.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:25.108 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--endpoints', 'http://smithi193.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:25.108 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --endpoints http://smithi193.front.sepia.ceph.com:8001 --tier-type pubsub --sync-from-all=0 --sync-from test-zone1 --tier-config start_with_full_sync=false,event_retention_days=7 --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:55:26.388 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:55:26.391 7fe575eff840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T13:55:26.389 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:55:26.391 7fe575eff840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T13:55:26.453 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:55:26.456 7fe575eff840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T13:55:26.453 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 13:55:26.456 7fe575eff840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T13:55:26.520 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","domain_root":"test-zone4.rgw.meta:root","control_pool":"test-zone4.rgw.control","gc_pool":"test-zone4.rgw.log:gc","lc_pool":"test-zone4.rgw.log:lc","log_pool":"test-zone4.rgw.log","intent_log_pool":"test-zone4.rgw.log:intent","usage_log_pool":"test-zone4.rgw.log:usage","reshard_pool":"test-zone4.rgw.log:reshard","user_keys_pool":"test-zone4.rgw.meta:users.keys","user_email_pool":"test-zone4.rgw.meta:users.email","user_swift_pool":"test-zone4.rgw.meta:users.swift","user_uid_pool":"test-zone4.rgw.meta:users.uid","otp_pool":"test-zone4.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone4.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone4.rgw.buckets.data"}},"data_extra_pool":"test-zone4.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","tier_config":{"event_retention_days":7,"start_with_full_sync":false},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T13:55:26.521 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool create test-zone4.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:55:28.121 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'test-zone4.rgw.buckets.data' created 2020-05-24T13:55:28.135 INFO:teuthology.orchestra.run.smithi193:> sudo ceph osd pool application enable test-zone4.rgw.buckets.data rgw --cluster c2 2020-05-24T13:55:29.142 INFO:teuthology.orchestra.run.smithi193.stderr:enabled application 'rgw' on pool 'test-zone4.rgw.buckets.data' 2020-05-24T13:55:29.152 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:29.153 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:29.153 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:55:35.870 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","domain_root":"test-zone4.rgw.meta:root","control_pool":"test-zone4.rgw.control","gc_pool":"test-zone4.rgw.log:gc","lc_pool":"test-zone4.rgw.log:lc","log_pool":"test-zone4.rgw.log","intent_log_pool":"test-zone4.rgw.log:intent","usage_log_pool":"test-zone4.rgw.log:usage","reshard_pool":"test-zone4.rgw.log:reshard","user_keys_pool":"test-zone4.rgw.meta:users.keys","user_email_pool":"test-zone4.rgw.meta:users.email","user_swift_pool":"test-zone4.rgw.meta:users.swift","user_uid_pool":"test-zone4.rgw.meta:users.uid","otp_pool":"test-zone4.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone4.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone4.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone4.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","tier_config":{"event_retention_days":7,"start_with_full_sync":false},"realm_id":""} 2020-05-24T13:55:35.871 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:35.871 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:35.872 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:56:05.166 INFO:teuthology.orchestra.run.smithi193.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T13:56:08.256 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","epoch":4,"predecessor_uuid":"9f6e589d-e881-43af-8021-1a7b7a957375","sync_status":[],"period_map":{"id":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:56:08.258 DEBUG:tasks.rgw.c2.client.1:waiting for process to exit 2020-05-24T13:56:08.258 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:56:08.276 INFO:tasks.rgw.c2.client.1.smithi193.stdout:2020-05-24 13:56:08.266 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground (PID: 13501) UID: 0 2020-05-24T13:56:08.280 INFO:tasks.rgw.c2.client.1.smithi193.stdout:2020-05-24 13:56:08.270 4049040 -1 shutting down 2020-05-24T13:56:14.360 INFO:tasks.rgw.c2.client.1:Stopped 2020-05-24T13:56:14.361 INFO:tasks.rgw.c2.client.1:Restarting daemon 2020-05-24T13:56:14.362 INFO:teuthology.orchestra.run.smithi193:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2020-05-24T13:56:14.365 INFO:tasks.rgw.c2.client.1:Started 2020-05-24T13:56:14.365 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8001 2020-05-24T13:56:14.412 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:56:14.412 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:56:14.419 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:56:14.420 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:56:15.421 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8001 2020-05-24T13:56:15.467 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:56:15.467 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:56:15.471 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:56:15.471 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:56:17.473 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8001 2020-05-24T13:56:17.525 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:56:17.525 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:56:17.528 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:56:17.530 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:56:21.531 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8001 2020-05-24T13:56:21.584 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:56:21.585 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:56:21.585 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi193.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:56:21.586 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:56:29.589 INFO:teuthology.orchestra.run.smithi193:> curl http://smithi193.front.sepia.ceph.com:8001 2020-05-24T13:56:29.635 INFO:teuthology.orchestra.run.smithi193.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:56:29.636 INFO:teuthology.orchestra.run.smithi193.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:56:29.765 INFO:teuthology.orchestra.run.smithi193.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 134 100 134 0 0 1035 0 --:--:-- --:--:-- --:--:-- 1038 2020-05-24T13:56:29.765 INFO:teuthology.orchestra.run.smithi193.stdout:{"Code":"MethodNotAllowed","RequestId":"tx000000000000000000001-005eca7d0d-1111-test-zone4","HostId":"1111-test-zone4-test-zonegroup"} 2020-05-24T13:56:29.766 INFO:tasks.rgw_multisite:rgw multisite configuration completed 2020-05-24T13:56:29.767 INFO:teuthology.run_tasks:Running task rgw-multisite-tests... 2020-05-24T13:56:29.859 INFO:tasks.rgw_multisite_tests:creating test user.. 2020-05-24T13:56:29.859 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:56:29.860 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:56:29.860 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid rgw-multisite-test-user --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Multisite Test User' --gen-access-key --gen-secret --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:56:33.213 INFO:teuthology.orchestra.run.smithi019.stdout:{"user_id":"rgw-multisite-test-user","display_name":"Multisite Test User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"rgw-multisite-test-user","access_key":"JTYB2R5VIDF0E7ZS3QCH","secret_key":"3ywpWH5iBFnl6Q9p2MYpdzjMAyVAteLY7X6rs5g6"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2020-05-24T13:56:34.032 INFO:rgw_multi.tests:meta checkpoint 2020-05-24T13:56:34.033 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:34.033 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:34.034 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T13:56:37.563 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328592.917777_55.1","last_update":"2020-05-24 13:56:32.917777Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:56:37.564 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:56:37.564 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:37.565 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:37.578 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:40.638 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.508687_53.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.508687Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:40.639 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1590328592.917777_55.1 target=1_1590328592.508687_53.1 2020-05-24T13:56:45.640 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:45.641 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:45.642 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:48.244 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:48.244 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:56:48.245 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:56:48.245 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:48.245 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:48.246 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:51.177 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:51.177 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328592.917777_55.1 target= 2020-05-24T13:56:56.178 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:56.179 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:56.180 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:59.020 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:59.021 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328592.917777_55.1 target= 2020-05-24T13:57:04.022 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:04.023 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:04.024 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:06.664 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:06.666 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328592.917777_55.1 target= 2020-05-24T13:57:11.667 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:11.668 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:11.669 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:14.816 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:14.817 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328592.917777_55.1 target= 2020-05-24T13:57:19.819 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:19.820 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:19.821 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:22.371 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:22.372 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328592.917777_55.1 target= 2020-05-24T13:57:27.373 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:27.374 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:27.375 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:30.568 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:30.569 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328592.917777_55.1 target= 2020-05-24T13:57:35.570 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:35.571 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:35.572 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:38.576 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:38.577 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328592.917777_55.1 target= 2020-05-24T13:57:43.579 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:43.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:43.581 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:47.139 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:47.140 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:57:47.141 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:57:47.141 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:47.141 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:47.142 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:47.291 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:08:57.516 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T13:57:47.402 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:08:57.627 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T13:57:48.049 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:08:58.274 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T13:57:49.670 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:49.670 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:57:49.671 INFO:rgw_multi.tests:meta checkpoint 2020-05-24T13:57:49.671 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:49.671 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:49.672 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:51.601 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T13:57:52.389 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328592.917777_55.1","last_update":"2020-05-24 13:56:32.917777Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:57:52.389 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:57:52.390 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:52.390 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:52.391 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:54.989 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:54.990 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:57:54.990 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:57:54.991 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:54.991 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:54.991 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:58.306 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:58.307 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:57:58.307 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:57:58.308 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:58.309 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:58.309 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:00.723 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:00.724 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:58:00.724 INFO:tasks.rgw_multisite_tests:running rgw multisite tests on 'tasks.rgw_multi.tests' with args=[] 2020-05-24T13:58:00.732 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-1 2020-05-24T13:58:14.034 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-2 2020-05-24T13:58:32.202 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-3 2020-05-24T13:58:33.718 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:33.719 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:33.720 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:35.678 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T13:58:36.335 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328592.917777_55.1","last_update":"2020-05-24 13:56:32.917777Z"},{"marker":"1_1590328712.981332_45.1","last_update":"2020-05-24 13:58:32.981332Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:58:36.336 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:58:36.336 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:36.337 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:36.338 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:38.703 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328702.912912_41.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:58:22.912912Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:38.704 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1590328712.981332_45.1 target=1_1590328702.912912_41.1 2020-05-24T13:58:43.706 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:43.707 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:43.707 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:46.282 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328702.912912_41.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:58:22.912912Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:46.282 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1590328712.981332_45.1 target=1_1590328702.912912_41.1 2020-05-24T13:58:51.283 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:51.285 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:51.286 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:53.714 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328712.981332_45.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:58:32.981332Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:53.716 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:58:53.716 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:58:53.717 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:53.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:53.718 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:54.086 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:08:57.278 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T13:58:54.193 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:08:57.385 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T13:58:54.856 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:08:58.048 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T13:58:56.445 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328712.981332_45.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:58:32.981332Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:56.446 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:58:56.447 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:58:56.447 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:56.448 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:56.448 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:58.950 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328712.981332_45.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:58:32.981332Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:58.951 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:59:00.107 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_create ... ok 2020-05-24T13:59:00.109 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-4 2020-05-24T13:59:00.797 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-5 2020-05-24T13:59:02.727 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-6 2020-05-24T13:59:04.062 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:04.063 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:04.064 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:06.263 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T13:59:06.852 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328592.917777_55.1","last_update":"2020-05-24 13:56:32.917777Z"},{"marker":"1_1590328743.504813_57.1","last_update":"2020-05-24 13:59:03.504813Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:59:06.852 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:59:06.853 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:06.854 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:06.854 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:09.489 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328743.504813_57.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:03.504813Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:09.490 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:59:09.490 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:59:09.491 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:09.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:09.492 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:12.496 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328743.504813_57.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:03.504813Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:12.497 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:59:12.498 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:59:12.498 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:12.499 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:12.499 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:15.312 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328743.504813_57.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:03.504813Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:15.315 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:59:31.172 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:31.174 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:31.175 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:32.938 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T13:59:33.738 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328592.917777_55.1","last_update":"2020-05-24 13:56:32.917777Z"},{"marker":"1_1590328769.600141_111.1","last_update":"2020-05-24 13:59:29.600141Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:59:33.738 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:59:33.739 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:33.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:33.740 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:36.133 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328769.600141_111.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:29.600141Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:36.134 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:59:36.135 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:59:36.135 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:36.136 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:36.136 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:39.274 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328769.600141_111.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:29.600141Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:39.274 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:59:39.275 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:59:39.275 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:39.276 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:39.277 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:42.035 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328769.600141_111.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:29.600141Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:42.035 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:59:43.103 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_recreate ... ok 2020-05-24T13:59:43.105 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-7 2020-05-24T13:59:43.945 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-8 2020-05-24T13:59:45.569 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-9 2020-05-24T13:59:47.021 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:47.022 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:47.022 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:48.910 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T13:59:49.506 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328592.917777_55.1","last_update":"2020-05-24 13:56:32.917777Z"},{"marker":"1_1590328786.554014_123.1","last_update":"2020-05-24 13:59:46.554014Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:59:49.507 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:59:49.508 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:49.509 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:49.509 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:51.922 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328786.554014_123.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:46.554014Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:51.923 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:59:51.924 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:59:51.925 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:51.925 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:51.926 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:54.800 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328786.554014_123.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:46.554014Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:54.801 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:59:54.802 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:59:54.802 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:54.803 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:54.804 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:57.542 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328786.554014_123.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 13:59:46.554014Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:57.543 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:00:02.074 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:02.075 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:02.076 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:00:04.754 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328592.917777_55.1","last_update":"2020-05-24 13:56:32.917777Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:00:04.755 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:00:04.755 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:04.756 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:04.757 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:07.362 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:07.363 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:00:07.363 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:00:07.364 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:07.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:07.365 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:10.190 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:10.191 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:00:10.191 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:00:10.192 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:10.192 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:10.193 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:12.908 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328592.917777_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:32.917777Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:12.909 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:00:13.735 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_remove ... ok 2020-05-24T14:00:13.737 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-10 2020-05-24T14:00:14.584 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-11 2020-05-24T14:00:16.459 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-12 2020-05-24T14:00:27.820 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:27.821 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:27.822 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:33.448 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:00:34.309 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590328817.073639_76.1","last_update":"2020-05-24 14:00:17.073639Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:00:34.310 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:00:34.310 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:34.311 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:34.312 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:37.126 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328817.073639_76.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:00:17.073639Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:37.127 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:00:37.128 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:00:37.129 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:37.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:37.130 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:40.294 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328817.073639_76.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:00:17.073639Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:40.295 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:00:40.296 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:00:40.296 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:40.299 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:40.300 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:42.913 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328817.073639_76.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:00:17.073639Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:42.914 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:00:42.915 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:42.916 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:42.917 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:45.967 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.8.5"}]} 2020-05-24T14:00:45.968 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-10 2020-05-24T14:00:45.969 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:45.969 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:45.972 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-10 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:48.503 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.8.5"}}}] 2020-05-24T14:00:48.504 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-10 2020-05-24T14:00:48.505 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-10 zones={test-zone2, test-zone1} 2020-05-24T14:00:48.726 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:00:48.882 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:48.883 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:48.883 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:48.884 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:48.884 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:00:49.047 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:49.047 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:49.056 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:49.056 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:49.386 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:49.850 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:50.330 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:50.762 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:51.057 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:51.404 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:51.663 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:51.999 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:52.538 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:52.749 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:52.990 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:53.445 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:53.738 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-10 zones={test-zone2, test-zone1} 2020-05-24T14:00:53.739 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:53.740 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:53.741 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:56.284 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.8.5"}]} 2020-05-24T14:00:56.284 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-10 2020-05-24T14:00:56.285 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:56.286 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:56.287 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:58.844 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.8.5"}}}] 2020-05-24T14:00:58.844 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-10 2020-05-24T14:00:58.845 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-10 zones={test-zone3, test-zone1} 2020-05-24T14:00:58.951 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:00:59.035 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:59.036 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:59.036 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:59.037 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:59.037 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:00:59.108 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:59.109 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:59.109 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:59.110 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:59.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:59.482 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:59.829 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:01:00.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:00.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:00.674 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:00.901 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:01.148 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:01.483 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:01.778 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:01.992 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:02.379 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:02.668 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-10 zones={test-zone3, test-zone1} 2020-05-24T14:01:02.668 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:02.669 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:02.669 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.8.5"}]} 2020-05-24T14:01:05.146 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-10 2020-05-24T14:01:05.147 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:05.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:05.148 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-10 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:07.910 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.8.5"}}}] 2020-05-24T14:01:07.910 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-10 2020-05-24T14:01:07.911 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:07.912 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:07.912 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:10.483 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.11.5"}]} 2020-05-24T14:01:10.483 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-11 2020-05-24T14:01:10.484 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:10.484 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:10.485 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:13.621 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.11.5"}}}] 2020-05-24T14:01:13.622 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-11 2020-05-24T14:01:13.623 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-11 zones={test-zone1, test-zone2} 2020-05-24T14:01:13.789 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:01:13.918 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:01:13.919 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:01:13.920 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:01:13.921 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:01:13.921 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:01:14.003 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:01:14.003 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:01:14.004 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:01:14.004 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:01:14.334 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:01:14.738 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:01:15.250 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:01:15.597 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:15.835 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:16.320 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:16.607 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:16.893 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:17.274 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:17.555 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:17.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:18.258 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:18.660 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-11 zones={test-zone1, test-zone2} 2020-05-24T14:01:18.661 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:18.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:18.662 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:21.147 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.11.5"}]} 2020-05-24T14:01:21.148 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-11 2020-05-24T14:01:21.149 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:21.149 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:21.150 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:24.417 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.11.5"}}}] 2020-05-24T14:01:24.419 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-11 2020-05-24T14:01:24.419 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-11 zones={test-zone3, test-zone2} 2020-05-24T14:01:24.557 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:01:24.698 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:01:24.699 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:01:24.699 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:01:24.700 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:01:24.700 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:01:24.834 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:01:24.835 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:01:24.835 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:01:24.837 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:01:25.017 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:01:25.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:01:25.754 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:01:25.998 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:26.235 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:26.710 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:01:26.963 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:27.222 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:27.568 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:01:27.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:28.206 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:28.784 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:01:29.018 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-11 zones={test-zone3, test-zone2} 2020-05-24T14:01:29.019 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:29.019 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:29.020 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:31.259 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.11.5"}]} 2020-05-24T14:01:31.260 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-11 2020-05-24T14:01:31.261 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:31.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:31.262 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:33.694 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:33.694 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:01:38.696 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:38.697 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:38.697 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:41.325 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:41.326 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:01:46.326 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:46.327 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:46.328 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:49.083 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:49.084 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:01:54.088 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:54.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:54.090 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:56.822 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:56.823 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:01.825 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:01.826 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:01.827 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:04.685 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:04.686 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:09.688 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:09.689 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:09.690 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:12.382 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:12.383 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:17.385 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:17.386 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:17.387 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:20.002 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:20.003 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:25.004 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:25.005 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:25.006 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:27.790 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:27.791 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:32.792 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:32.793 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:32.794 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:35.769 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:35.770 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:40.771 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:40.772 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:40.773 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:43.673 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:43.674 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:48.676 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:48.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:48.678 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:51.387 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:51.388 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:02:56.390 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:56.391 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:56.392 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:58.974 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:58.975 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:03.977 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:03.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:03.978 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:06.618 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:06.619 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:11.621 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:11.622 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:11.623 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:14.309 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:14.310 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:19.311 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:19.312 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:19.313 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:22.113 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:22.114 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:27.114 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:27.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:27.116 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:29.918 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:29.919 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:34.920 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:34.921 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:34.922 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:37.610 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:37.611 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:42.612 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:42.613 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:42.614 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:45.295 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:45.296 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:50.297 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:50.298 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:50.298 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:53.051 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:53.052 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:03:58.053 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:58.054 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:58.055 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:00.793 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:00.793 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:04:05.794 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:05.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:05.797 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:08.658 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:08.659 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:04:13.660 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:13.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:13.662 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:16.095 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:16.095 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:04:21.096 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:21.097 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:21.098 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:23.917 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:23.955 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:04:28.956 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:28.957 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:28.958 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:31.794 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:31.795 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:04:36.796 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:36.798 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:36.799 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:39.590 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:39.591 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:04:44.596 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:44.597 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:44.598 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:47.218 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:47.219 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:04:52.221 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:52.222 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:52.223 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:55.340 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:55.341 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:00.347 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:00.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:00.349 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:03.226 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:03.227 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:08.228 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:08.229 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:08.230 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:11.022 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:11.023 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:16.025 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:16.026 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:16.027 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:18.689 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:18.690 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:23.691 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:23.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:23.692 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:26.362 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:26.363 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:31.365 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:31.366 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:31.367 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:34.194 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:34.195 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:39.203 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:39.204 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:39.211 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:40.891 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:16:51.120 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:40.898 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:16:51.128 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:41.367 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:16:51.597 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:42.134 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:42.135 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:47.137 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:47.138 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:47.139 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:50.223 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:50.223 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:05:55.225 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:55.226 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:55.227 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:58.182 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:58.183 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:03.185 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:03.186 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:03.187 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:05.971 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:05.972 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:08.149 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:17:18.379 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:08.152 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:17:18.382 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:08.184 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:17:18.414 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:10.973 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:10.974 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:10.975 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:14.124 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:14.125 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:19.126 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:19.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:19.128 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:22.121 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:22.121 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:27.122 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:27.123 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:27.124 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:30.652 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:30.653 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:35.654 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:35.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:35.656 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:38.513 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:38.514 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:43.516 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:43.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:43.518 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:46.239 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:46.240 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:51.242 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:51.244 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:51.244 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:53.940 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:53.941 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:06:58.943 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:58.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:58.945 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:01.032 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:17:04.231 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:07:01.037 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:17:04.236 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:07:01.387 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:17:04.587 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:07:01.389 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:01.390 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:07:06.391 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:06.392 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:06.393 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:08.943 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:08.944 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:07:13.945 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:13.946 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:13.947 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:14.732 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:17:17.931 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:07:14.733 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:17:17.933 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:07:14.756 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:17:17.956 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:07:16.454 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:16.455 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:07:21.458 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:21.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:21.460 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:24.298 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:24.298 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:07:29.299 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:29.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:29.301 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:32.184 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:32.185 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:07:37.187 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:37.188 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:37.189 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:39.931 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:39.932 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:07:44.933 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:44.934 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:44.935 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:47.757 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:47.758 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:07:52.759 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:52.759 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:52.760 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:55.558 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:55.558 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:00.560 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:00.561 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:00.561 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:03.447 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:03.448 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:08.449 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:08.450 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:08.451 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:11.217 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:11.217 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:16.219 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:16.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:16.221 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:19.247 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:19.248 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:24.249 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:24.250 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:24.251 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:26.784 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:26.784 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:31.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:31.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:31.789 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:34.552 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:34.552 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:39.554 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:39.555 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:39.556 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:42.304 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:42.305 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:47.307 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:47.308 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:47.309 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:50.149 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:50.150 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:08:55.151 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:55.152 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:55.153 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:57.835 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:57.836 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:09:02.837 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:02.838 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:02.839 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:05.513 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:09:05.514 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:09:10.516 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:10.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:10.517 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:13.168 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:09:13.169 WARNING:rgw_multi.tests:bucket fwzbiz-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.11.5 target= 2020-05-24T14:09:18.171 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_object_sync ... FAIL 2020-05-24T14:09:18.172 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-13 2020-05-24T14:09:19.063 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-14 2020-05-24T14:09:20.897 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-15 2020-05-24T14:09:23.922 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:23.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:23.924 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:09:30.491 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590329361.625971_108.1","last_update":"2020-05-24 14:09:21.625971Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:09:30.492 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:09:30.492 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:30.493 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:30.493 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:32.877 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590329361.625971_108.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:09:21.625971Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:09:32.878 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:09:32.879 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:09:32.879 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:32.880 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:32.880 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:35.688 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590329361.625971_108.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:09:21.625971Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:09:35.689 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:09:35.690 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:09:35.690 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:35.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:35.691 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:38.576 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590329361.625971_108.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:09:21.625971Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:09:38.576 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:09:38.577 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:38.577 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:38.578 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000002.4.5"}]} 2020-05-24T14:09:41.452 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-13 2020-05-24T14:09:41.453 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:41.453 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:41.454 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:44.093 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.4.5"}}}] 2020-05-24T14:09:44.094 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-13 2020-05-24T14:09:44.094 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-13 zones={test-zone2, test-zone1} 2020-05-24T14:09:44.197 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:09:44.255 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:44.256 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:09:44.318 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:44.564 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:44.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:45.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:45.622 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-13 zones={test-zone2, test-zone1} 2020-05-24T14:09:45.623 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:45.624 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:45.625 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:48.343 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000002.4.5"}]} 2020-05-24T14:09:48.344 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-13 2020-05-24T14:09:48.344 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:48.345 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:48.346 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:51.101 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.4.5"}}}] 2020-05-24T14:09:51.102 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-13 2020-05-24T14:09:51.103 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-13 zones={test-zone3, test-zone1} 2020-05-24T14:09:51.202 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:09:51.257 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:51.258 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:09:51.324 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:51.500 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:51.766 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:52.249 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:52.496 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-13 zones={test-zone3, test-zone1} 2020-05-24T14:09:52.497 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:52.497 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:52.498 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:55.037 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000002.4.5"}]} 2020-05-24T14:09:55.038 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-13 2020-05-24T14:09:55.039 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:55.039 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:55.040 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:57.816 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.4.5"}}}] 2020-05-24T14:09:57.817 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-13 2020-05-24T14:09:57.817 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:57.818 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:57.819 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:00.531 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T14:10:00.531 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-14 2020-05-24T14:10:00.532 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:00.533 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:00.533 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:03.410 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.3.5"}}}] 2020-05-24T14:10:03.410 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-14 2020-05-24T14:10:03.411 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-14 zones={test-zone1, test-zone2} 2020-05-24T14:10:03.632 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:10:03.706 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:10:03.706 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:10:03.796 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:10:04.048 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:10:04.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:10:04.906 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:10:05.258 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-14 zones={test-zone1, test-zone2} 2020-05-24T14:10:05.259 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:05.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:05.262 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:08.005 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T14:10:08.006 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-14 2020-05-24T14:10:08.006 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:08.007 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:08.007 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:11.020 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.3.5"}}}] 2020-05-24T14:10:11.021 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-14 2020-05-24T14:10:11.022 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-14 zones={test-zone3, test-zone2} 2020-05-24T14:10:11.174 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:10:11.250 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:10:11.251 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:10:11.340 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:10:11.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:10:11.816 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:10:12.204 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:10:12.440 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-14 zones={test-zone3, test-zone2} 2020-05-24T14:10:12.442 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:12.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:12.443 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:14.996 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T14:10:14.997 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-14 2020-05-24T14:10:14.997 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:14.998 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:14.998 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:17.802 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:17.803 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:22.805 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:22.806 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:22.807 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:25.535 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:25.536 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:30.537 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:30.538 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:30.539 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:33.245 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:33.246 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:38.247 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:38.248 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:38.249 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:41.066 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:41.067 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:46.068 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:46.069 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:46.070 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:48.815 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:48.816 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:53.817 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:53.818 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:53.819 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:56.408 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:56.409 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:01.411 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:01.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:01.412 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:04.435 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:04.436 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:09.437 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:09.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:09.439 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:12.409 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:12.409 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:17.410 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:17.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:17.412 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:20.416 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:20.417 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:25.419 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:25.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:25.421 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:27.910 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:27.911 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:32.912 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:32.912 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:32.913 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:35.690 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:35.691 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:40.692 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:40.693 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:40.694 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:43.339 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:43.340 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:48.342 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:48.343 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:48.344 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:51.276 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:51.277 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:56.279 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:56.280 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:56.281 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:59.234 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:59.235 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:04.236 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:04.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:04.238 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:06.792 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:06.793 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:11.794 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:11.795 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:11.795 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:14.507 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:14.508 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:19.510 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:19.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:19.512 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:22.405 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:22.405 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:27.407 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:27.408 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:27.408 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:29.932 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:29.932 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:34.933 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:34.934 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:34.935 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:36.350 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:23:46.579 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:12:36.353 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:23:46.582 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:12:36.659 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:23:46.888 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:12:37.432 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:37.433 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:42.434 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:42.435 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:42.436 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:45.206 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:45.207 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:50.209 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:50.210 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:50.211 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:52.895 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:52.895 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:57.897 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:57.898 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:57.899 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:00.734 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:00.736 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:05.738 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:05.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:05.739 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:08.435 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:08.436 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:13.437 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:13.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:13.439 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:16.265 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:16.266 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:21.267 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:21.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:21.278 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:24.086 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:24.087 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:29.088 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:29.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:29.090 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:32.329 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:32.376 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:37.377 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:37.378 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:37.379 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:40.044 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:40.044 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:45.046 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:45.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:45.047 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:47.877 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:47.877 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:52.879 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:52.880 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:52.880 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:55.783 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:55.784 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:56.550 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:23:59.741 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:13:56.613 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:23:59.805 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:13:56.916 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:24:00.108 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:14:00.785 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:00.786 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:00.786 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:03.403 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:03.404 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:08.406 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:08.407 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:08.408 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:11.336 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:11.336 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:16.338 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:16.339 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:16.339 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:19.413 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:19.414 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:24.415 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:24.416 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:24.416 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:26.964 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:26.965 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:28.985 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:25:39.212 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:14:28.987 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:25:39.214 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:14:29.055 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:25:39.282 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:14:31.966 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:31.967 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:31.968 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:34.782 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:34.783 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:39.785 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:39.786 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:39.787 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:42.534 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:42.534 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:47.536 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:47.536 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:47.537 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:50.300 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:50.301 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:55.302 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:55.303 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:55.304 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:58.188 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:58.188 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:03.190 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:03.191 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:03.192 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:05.821 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:05.821 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:10.824 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:10.825 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:10.825 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:13.554 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:13.555 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:18.556 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:18.557 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:18.558 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:21.211 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:21.212 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:26.214 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:26.215 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:26.215 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:29.088 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:29.089 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:34.090 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:34.091 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:34.092 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:35.365 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:25:38.554 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:15:35.378 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:25:38.567 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:15:35.509 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:25:38.698 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:15:36.905 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:36.906 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:41.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:41.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:41.909 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:44.787 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:44.788 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:49.789 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:49.790 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:49.790 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:52.540 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:52.541 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:57.542 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:57.542 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:57.543 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:00.344 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:00.345 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:05.346 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:05.347 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:05.347 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:07.610 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:07.611 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:12.612 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:12.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:12.614 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:15.583 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:15.584 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:20.585 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:20.586 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:20.587 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:23.436 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:23.436 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:28.438 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:28.439 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:28.439 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:32.029 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:32.030 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:37.031 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:37.032 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:37.033 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:40.126 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:40.127 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:45.128 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:45.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:45.129 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:47.727 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:47.728 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:52.729 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:52.730 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:52.731 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:55.459 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:55.460 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:00.462 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:00.462 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:00.463 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:03.297 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:03.298 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:08.299 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:08.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:08.301 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:11.245 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:11.245 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:16.247 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:16.247 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:16.248 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:18.927 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:18.928 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:23.929 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:23.930 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:23.930 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:26.591 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:26.592 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:31.593 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:31.594 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:31.595 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:34.342 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:34.342 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:39.344 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:39.345 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:39.346 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:42.348 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:42.349 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:47.350 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:47.351 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:47.352 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:50.413 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:50.414 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:55.415 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:55.416 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:55.416 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:58.155 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:58.155 WARNING:rgw_multi.tests:bucket fwzbiz-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:18:03.157 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_object_delete ... FAIL 2020-05-24T14:18:03.158 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-16 2020-05-24T14:18:04.222 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-17 2020-05-24T14:18:06.151 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-18 2020-05-24T14:18:09.826 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:09.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:09.828 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:18:12.677 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590329889.537905_171.1","last_update":"2020-05-24 14:18:09.537905Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:18:12.678 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:18:12.678 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:12.679 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:12.679 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:15.051 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590329889.537905_171.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:18:09.537905Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:18:15.052 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:18:15.052 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:18:15.053 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:15.053 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:15.054 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:17.920 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590329889.537905_171.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:18:09.537905Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:18:17.921 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:18:17.922 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:18:17.922 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:17.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:17.923 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:20.809 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590329889.537905_171.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:18:09.537905Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:18:20.810 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:18:21.862 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:21.863 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:21.864 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:25.866 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000003.5.10"}]} 2020-05-24T14:18:25.867 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-16 2020-05-24T14:18:25.868 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:25.868 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:25.869 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-16 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:28.901 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"oyPU1ZlV9f53PZjTXY57fj9Kha0-DOR","ns":""},"count":1},"inc_marker":{"position":"00000000007.12.9"}}}] 2020-05-24T14:18:28.902 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-16 2020-05-24T14:18:28.902 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:28.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:28.903 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:31.783 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000007.12.9"}]} 2020-05-24T14:18:31.783 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-16 2020-05-24T14:18:31.784 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:31.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:31.785 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:34.521 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"oyPU1ZlV9f53PZjTXY57fj9Kha0-DOR","ns":""},"count":1},"inc_marker":{"position":"00000000007.12.9"}}}] 2020-05-24T14:18:34.522 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-16 2020-05-24T14:18:34.523 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:34.523 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:34.524 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:37.357 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000007.12.9"}]} 2020-05-24T14:18:37.357 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-16 2020-05-24T14:18:37.358 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:37.359 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:37.359 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-16 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:40.479 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"oyPU1ZlV9f53PZjTXY57fj9Kha0-DOR","ns":""},"count":1},"inc_marker":{"position":"00000000007.12.9"}}}] 2020-05-24T14:18:40.480 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-16 2020-05-24T14:18:41.262 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:41.263 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:41.264 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:43.845 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000003.5.10"}]} 2020-05-24T14:18:43.846 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-17 2020-05-24T14:18:43.846 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:43.847 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:43.848 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:48.578 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"fn3kaViefyZEr6z7rI-iwAAZPoLH0ti","ns":""},"count":1},"inc_marker":{"position":"00000000005.9.9"}}}] 2020-05-24T14:18:48.579 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-17 2020-05-24T14:18:48.579 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:48.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:48.580 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:50.935 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000006.11.9"}]} 2020-05-24T14:18:50.936 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-17 2020-05-24T14:18:50.937 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:50.937 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:50.938 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:54.122 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"fn3kaViefyZEr6z7rI-iwAAZPoLH0ti","ns":""},"count":1},"inc_marker":{"position":"00000000006.11.9"}}}] 2020-05-24T14:18:54.123 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-17 2020-05-24T14:18:54.123 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:54.124 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:54.125 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:56.654 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000006.11.9"}]} 2020-05-24T14:18:56.655 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-17 2020-05-24T14:18:56.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:56.656 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:56.656 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:59.487 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:18:59.488 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:04.490 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:04.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:04.492 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:07.292 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:07.293 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:12.295 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:12.296 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:12.297 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:15.223 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:15.223 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:20.225 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:20.226 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:20.226 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:22.941 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:22.941 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:27.947 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:27.948 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:27.949 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:30.935 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:30.936 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:31.704 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:30:41.928 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:19:31.707 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:30:41.932 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:19:31.950 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:30:42.172 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:19:35.938 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:35.939 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:35.940 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:38.796 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:38.796 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:43.797 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:43.798 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:43.799 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:46.724 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:46.725 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:51.726 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:51.727 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:51.728 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:54.536 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:54.537 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:19:59.538 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:59.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:59.540 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:02.111 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:02.111 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:20:07.113 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:07.113 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:07.114 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:09.826 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:09.827 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:20:14.828 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:14.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:14.830 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:17.473 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:17.473 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:20:22.475 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:22.476 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:22.477 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:25.291 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:25.291 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:20:30.293 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:30.294 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:30.295 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:33.055 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:33.055 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:20:38.057 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:38.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:38.058 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:41.184 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:41.184 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:20:46.186 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:46.187 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:46.187 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:49.037 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:49.038 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:20:51.872 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:30:55.054 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:20:51.874 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:30:55.057 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:20:52.138 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:30:55.320 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:20:54.039 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:54.040 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:54.041 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:57.116 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:57.117 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:02.118 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:02.119 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:02.121 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:04.893 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:04.894 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:09.895 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:09.896 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:09.896 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:12.968 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:12.969 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:17.971 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:17.972 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:17.973 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:20.838 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:20.839 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:25.840 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:25.841 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:25.842 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:28.582 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:28.582 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:33.583 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:33.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:33.585 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:36.445 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:36.446 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:41.447 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:41.448 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:41.449 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:44.624 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:44.625 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:49.627 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:49.628 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:49.629 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:52.184 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:52.185 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:21:57.186 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:57.187 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:57.188 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:59.797 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:59.798 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:22:04.799 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:04.800 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:04.801 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:07.784 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:07.785 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:22:12.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:12.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:12.788 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:16.134 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:16.134 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:22:21.136 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:21.137 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:21.138 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:24.329 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:24.329 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:22:29.330 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:29.331 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:29.332 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:32.239 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:32.240 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:22:37.241 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:37.242 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:37.243 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:40.062 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:40.063 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:22:45.064 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:45.065 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:45.066 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:47.967 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:47.968 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:22:52.969 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:52.970 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:52.970 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:55.822 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:55.823 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:00.825 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:00.825 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:00.826 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:03.470 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:03.470 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:08.472 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:08.473 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:08.473 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:11.338 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:11.338 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:16.340 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:16.341 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:16.342 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:19.112 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:19.112 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:24.114 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:24.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:24.116 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:26.940 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:26.940 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:31.942 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:31.943 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:31.943 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:34.759 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:34.759 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:39.761 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:39.762 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:39.763 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:42.825 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:42.826 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:47.827 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:47.828 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:47.829 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:50.762 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:50.763 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:23:55.764 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:55.766 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:55.767 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:58.771 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:58.772 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:03.773 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:03.774 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:03.775 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:06.596 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:06.597 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:11.599 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:11.599 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:11.600 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:14.973 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:14.974 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:19.976 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:19.977 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:19.978 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:22.980 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:22.981 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:27.983 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:27.984 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:27.985 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:30.896 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:30.897 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:35.899 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:35.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:35.900 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:38.803 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:38.804 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:43.805 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:43.806 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:43.806 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:47.039 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:47.040 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:52.042 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:52.043 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:52.043 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:54.800 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:54.801 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:24:59.802 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:59.803 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:59.803 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:02.612 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:02.613 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:25:07.615 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:07.616 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:07.617 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:10.739 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:10.740 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:25:15.741 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:15.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:15.743 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:18.626 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:18.626 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:25:23.628 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:23.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:23.630 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:26.860 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:26.861 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:25:31.862 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:31.863 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:31.864 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:34.905 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:34.905 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:25:39.906 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:39.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:39.908 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:42.855 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:42.855 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:25:47.857 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:47.858 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:47.859 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:50.849 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:50.850 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:25:55.851 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:55.852 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:55.853 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:58.823 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:58.824 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:26:03.825 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:03.826 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:03.827 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:06.884 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:06.885 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:26:11.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:11.888 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:11.889 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:14.834 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:14.835 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:26:19.836 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:19.837 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:19.838 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:22.896 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:22.897 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:26:27.899 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:27.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:27.901 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:31.548 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:31.549 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:26:36.551 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:36.552 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:36.553 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:39.759 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:39.760 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:26:44.761 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:44.762 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:44.763 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:47.584 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:47.584 WARNING:rgw_multi.tests:bucket fwzbiz-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000006.11.9 target= 2020-05-24T14:26:52.586 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_versioned_object_incremental_sync ... FAIL 2020-05-24T14:26:53.882 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:53.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:53.883 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:26:56.530 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590330413.778058_212.1","last_update":"2020-05-24 14:26:53.778058Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:26:56.530 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:26:56.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:56.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:56.532 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:59.004 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590330413.778058_212.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:26:53.778058Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:26:59.005 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:26:59.005 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:26:59.006 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:59.006 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:59.007 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:01.898 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590330413.778058_212.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:26:53.778058Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:27:01.899 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:27:01.899 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:27:01.900 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:01.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:01.901 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:05.060 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590330413.778058_212.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:26:53.778058Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:27:05.061 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:27:05.927 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:05.927 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:05.928 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000003.12.10"}]} 2020-05-24T14:27:09.896 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-19 2020-05-24T14:27:09.897 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:09.898 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:09.898 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:13.203 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Z3d3Q7PzJeUR7KWXKI0nR69G1P6HOFX","ns":""},"count":1},"inc_marker":{"position":"00000000007.19.9"}}}] 2020-05-24T14:27:13.204 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-19 2020-05-24T14:27:13.205 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:13.205 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:13.206 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:16.009 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000007.19.9"}]} 2020-05-24T14:27:16.010 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-19 2020-05-24T14:27:16.010 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:16.011 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:16.012 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:18.717 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Z3d3Q7PzJeUR7KWXKI0nR69G1P6HOFX","ns":""},"count":1},"inc_marker":{"position":"00000000007.19.9"}}}] 2020-05-24T14:27:18.718 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-19 2020-05-24T14:27:18.719 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:18.719 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:18.720 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:21.660 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000007.19.9"}]} 2020-05-24T14:27:21.660 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-19 2020-05-24T14:27:21.661 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:21.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:21.662 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:24.488 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Z3d3Q7PzJeUR7KWXKI0nR69G1P6HOFX","ns":""},"count":1},"inc_marker":{"position":"00000000007.19.9"}}}] 2020-05-24T14:27:24.489 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-19 2020-05-24T14:27:24.490 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:24.490 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:24.491 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:26.938 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000005.21.9"}]} 2020-05-24T14:27:26.939 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-19 2020-05-24T14:27:26.940 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:26.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:26.941 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:29.921 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Z3d3Q7PzJeUR7KWXKI0nR69G1P6HOFX","ns":""},"count":1},"inc_marker":{"position":"00000000005.21.9"}}}] 2020-05-24T14:27:29.922 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-19 2020-05-24T14:27:29.922 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:29.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:29.923 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:32.485 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000005.21.9"}]} 2020-05-24T14:27:32.485 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-19 2020-05-24T14:27:32.486 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:32.486 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:32.487 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:35.229 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Z3d3Q7PzJeUR7KWXKI0nR69G1P6HOFX","ns":""},"count":1},"inc_marker":{"position":"00000000005.21.9"}}}] 2020-05-24T14:27:35.229 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-19 2020-05-24T14:27:35.230 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:35.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:35.231 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:37.696 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000005.21.9"}]} 2020-05-24T14:27:37.697 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-19 2020-05-24T14:27:37.697 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:37.698 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:37.699 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:40.351 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:40.352 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:27:45.354 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:45.355 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:45.355 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:48.379 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:48.380 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:27:53.381 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:53.382 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:53.382 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:56.214 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:56.215 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:01.216 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:01.217 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:01.218 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:03.959 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:03.960 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:08.961 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:08.962 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:08.963 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:11.718 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:11.719 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:16.721 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:16.722 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:16.723 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:19.706 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:19.706 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:24.707 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:24.708 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:24.709 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:27.729 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:27.730 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:32.731 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:32.732 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:32.732 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:35.673 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:35.673 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:40.674 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:40.675 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:40.676 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:43.608 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:43.609 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:48.610 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:48.611 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:48.611 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:51.766 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:51.767 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:28:56.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:56.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:56.769 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:59.705 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:59.705 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:29:04.707 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:04.708 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:04.709 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:07.871 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:07.872 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:29:12.873 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:12.874 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:12.875 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:15.819 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:15.820 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:29:20.821 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:20.823 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:20.823 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:24.123 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:24.123 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:29:29.125 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:29.126 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:29.127 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:31.968 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:31.969 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:29:36.970 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:36.972 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:36.974 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:39.940 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:39.941 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:29:44.942 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:44.943 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:44.944 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:47.687 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:47.688 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:29:52.689 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:52.690 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:52.691 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:55.650 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:55.650 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:00.652 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:00.653 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:00.654 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:03.924 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:03.925 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:08.926 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:08.927 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:08.927 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:11.863 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:11.864 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:16.866 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:16.867 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:16.867 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:19.544 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:19.544 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:24.546 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:24.547 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:24.548 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:27.630 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:27.630 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:32.631 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:32.632 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:32.633 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:35.483 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:35.484 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:40.485 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:40.486 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:40.487 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:43.529 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:43.529 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:48.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:48.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:48.533 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:51.612 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:51.612 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:30:56.614 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:56.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:56.616 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:59.492 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:59.493 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:31:04.495 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:04.496 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:04.496 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:07.532 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:07.533 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:31:10.731 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:42:20.950 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:31:10.741 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:42:20.960 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:31:11.224 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:42:21.443 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:31:12.534 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:12.535 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:12.536 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:15.599 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:15.600 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:31:20.607 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:20.608 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:20.609 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:23.700 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:23.701 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:31:28.702 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:28.703 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:28.704 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:31.465 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:31.466 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:31:36.467 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:36.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:36.469 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:39.351 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:39.351 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:31:44.353 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:44.354 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:44.355 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:47.374 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:47.374 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:31:52.376 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:52.378 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:52.379 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:55.127 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:55.128 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:00.129 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:00.130 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:00.130 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:03.334 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:03.335 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:08.336 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:08.337 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:08.338 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:11.310 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:11.311 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:16.313 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:16.314 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:16.315 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:19.133 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:19.134 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:22.106 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:42:25.289 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:32:22.119 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:42:25.302 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:32:22.566 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:42:25.749 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:32:24.136 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:24.137 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:24.137 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:26.903 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:26.903 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:31.905 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:31.906 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:31.906 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:34.640 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:34.641 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:39.642 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:39.643 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:39.644 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:42.734 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:42.735 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:47.735 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:47.737 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:47.738 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:50.577 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:50.578 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:32:55.580 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:55.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:55.581 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:58.487 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:58.487 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:03.488 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:03.489 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:03.490 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:06.553 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:06.554 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:11.555 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:11.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:11.557 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:14.541 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:14.542 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:19.544 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:19.544 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:19.545 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:21.888 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:44:32.107 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:33:21.892 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:44:32.112 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:33:22.009 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:44:32.229 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:33:22.226 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:22.227 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:27.229 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:27.230 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:27.231 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:30.079 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:30.080 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:35.082 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:35.083 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:35.084 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:38.088 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:38.089 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:43.091 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:43.091 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:43.092 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:46.153 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:46.153 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:51.155 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:51.155 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:51.156 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:54.141 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:54.142 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:33:59.142 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:59.143 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:59.144 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:02.291 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:02.292 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:34:07.293 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:07.294 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:07.295 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:10.599 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:10.600 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:34:15.602 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:15.602 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:15.603 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:18.925 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:18.925 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:34:23.926 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:23.927 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:23.928 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:26.671 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:26.672 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:34:31.673 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:31.674 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:31.675 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:34.533 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:34.534 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:34:39.536 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:39.536 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:39.537 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:42.498 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:44:45.682 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:34:42.500 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:44:45.684 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:34:42.578 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:42.579 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:34:42.635 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:44:45.818 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:34:47.580 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:47.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:47.582 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:50.466 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:50.467 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:34:55.468 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:55.469 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:55.470 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:58.643 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:58.644 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:35:03.645 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:03.646 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:03.646 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:06.682 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:35:06.682 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:35:11.684 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:11.685 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:11.686 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:14.521 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:35:14.521 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:35:19.527 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:19.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:19.533 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:23.084 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:35:23.085 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:35:28.086 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:28.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:28.088 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:30.911 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:35:30.912 WARNING:rgw_multi.tests:bucket fwzbiz-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.21.9 target= 2020-05-24T14:35:35.915 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_concurrent_versioned_object_incremental_sync ... FAIL 2020-05-24T14:35:36.978 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:36.979 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:36.980 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:35:40.306 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590330936.762225_225.1","last_update":"2020-05-24 14:35:36.762225Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:35:40.306 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:35:40.307 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:40.307 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:40.308 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:42.938 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590330936.762225_225.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:35:36.762225Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:35:42.938 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:35:42.939 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:35:42.939 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:42.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:42.940 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:45.702 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590330936.762225_225.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:35:36.762225Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:35:45.703 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:35:45.703 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:35:45.704 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:45.704 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:45.705 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:48.474 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590330936.762225_225.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:35:36.762225Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:35:48.475 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:35:49.015 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:49.016 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:49.016 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:52.180 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:52.181 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-20 2020-05-24T14:35:52.182 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:52.182 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:52.183 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:55.331 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:35:55.332 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-20 2020-05-24T14:35:55.332 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:55.333 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:55.333 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:58.198 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-20 2020-05-24T14:35:58.198 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:58.199 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:58.200 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:01.032 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:36:01.033 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-20 2020-05-24T14:36:01.033 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:01.034 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:01.034 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:03.869 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:36:03.870 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-20 2020-05-24T14:36:03.875 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:03.876 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:03.876 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:06.654 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:36:06.655 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-20 2020-05-24T14:36:06.656 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:06.656 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:06.657 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:09.154 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:36:09.155 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-20 2020-05-24T14:36:09.156 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:09.156 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:09.157 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:12.038 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:36:12.039 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-20 2020-05-24T14:36:12.040 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:12.040 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:12.041 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:14.811 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:36:14.812 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-20 2020-05-24T14:36:14.813 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:14.813 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:14.814 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:18.236 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:36:18.236 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-20 2020-05-24T14:36:18.237 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:18.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:18.243 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:21.341 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:36:21.342 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-20 2020-05-24T14:36:21.343 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:21.343 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:21.344 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:24.293 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:24.294 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:29.296 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:29.298 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:29.298 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:33.212 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:33.213 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:38.215 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:38.215 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:38.216 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:41.466 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:41.467 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:46.467 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:46.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:46.469 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:49.529 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:49.530 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:54.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:54.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:54.532 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:57.440 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:57.440 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:02.442 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:02.443 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:02.443 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:05.805 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:05.805 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:10.807 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:10.808 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:10.809 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:13.730 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:13.731 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:18.733 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:18.734 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:18.735 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:21.982 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:21.983 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:26.984 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:26.985 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:26.986 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:30.028 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:30.034 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:35.036 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:35.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:35.037 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:37.916 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:37.916 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:42.918 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:42.920 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:42.920 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:45.925 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:45.926 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:50.927 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:50.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:50.929 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:54.169 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:54.169 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:59.171 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:59.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:59.172 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:02.133 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:02.142 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:07.144 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:07.145 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:07.145 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:10.252 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:10.253 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:15.255 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:15.256 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:15.256 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:18.263 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:18.264 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:23.266 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:23.266 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:23.267 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:26.734 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:26.735 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:31.736 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:31.737 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:31.738 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:34.742 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:34.743 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:39.744 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:39.745 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:39.746 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:42.672 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:42.672 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:47.675 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:47.676 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:47.677 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:50.727 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:50.728 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:55.729 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:55.730 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:55.732 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:58.446 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:58.447 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:03.449 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:03.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:03.450 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:06.614 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:06.615 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:11.616 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:11.617 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:11.618 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:14.620 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:14.621 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:19.622 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:19.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:19.624 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:22.766 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:22.766 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:27.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:27.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:27.770 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:30.714 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:30.715 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:31.676 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:50:41.895 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:31.679 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:50:41.898 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:31.945 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:50:42.164 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:35.716 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:35.718 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:35.718 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:38.973 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:38.974 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:43.975 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:43.976 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:43.976 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:47.132 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:47.133 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:52.136 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:52.137 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:52.138 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:55.032 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:55.033 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:00.034 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:00.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:00.036 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:03.070 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:03.070 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:08.072 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:08.073 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:08.073 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:10.939 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:10.939 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:15.941 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:15.942 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:15.943 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:18.504 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:51:28.722 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:18.524 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:51:28.743 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:18.618 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:51:28.837 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:19.035 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:19.036 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:24.038 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:24.039 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:24.040 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:26.954 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:26.955 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:31.957 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:31.958 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:31.959 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:35.153 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:35.154 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:40.155 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:40.156 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:40.157 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:42.935 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:50:46.119 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:42.936 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:50:46.121 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:43.150 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:50:46.333 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:43.216 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:43.217 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:48.219 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:48.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:48.221 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:51.343 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:51.344 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:56.345 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:56.346 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:56.347 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:59.431 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:59.432 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:04.433 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:04.434 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:04.435 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:07.638 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:07.639 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:12.640 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:12.641 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:12.642 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:15.918 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:15.919 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:20.921 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:20.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:20.924 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:24.161 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:24.162 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:29.163 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:29.164 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:29.165 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:32.351 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:32.352 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:37.148 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:51:40.330 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:41:37.152 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:51:40.337 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:41:37.233 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:51:40.418 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:41:37.353 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:37.354 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:37.355 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:40.658 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:40.659 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:45.660 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:45.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:45.662 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:48.580 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:48.581 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:53.583 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:53.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:53.585 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:56.477 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:56.478 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:01.480 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:01.481 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:01.482 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:04.694 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:04.695 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:09.695 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:09.696 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:09.697 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:12.865 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:12.866 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:17.868 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:17.877 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:17.878 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:20.973 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:20.974 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:25.978 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:25.979 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:25.980 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:29.187 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:29.187 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:34.188 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:34.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:34.190 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:37.050 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:37.051 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:42.052 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:42.053 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:42.054 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:44.851 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:44.852 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:49.859 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:49.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:49.860 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:53.042 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:53.043 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:58.044 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:58.045 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:58.046 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:01.628 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:01.628 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:06.634 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:06.635 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:06.636 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:09.968 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:09.969 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:14.970 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:14.971 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:14.972 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:18.019 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:18.020 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:23.021 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:23.022 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:23.023 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:26.118 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:26.119 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:31.120 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:31.121 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:31.121 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:34.602 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:34.603 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:39.604 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:39.605 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:39.605 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:42.925 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:42.926 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:47.927 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:47.928 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:47.928 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:50.999 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:51.000 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:56.002 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:56.002 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:56.003 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:59.078 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:59.078 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:44:04.080 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:04.081 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:04.081 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:07.336 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:44:07.336 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:44:12.337 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:12.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:12.339 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:15.083 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:44:15.084 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:44:20.085 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:20.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:20.087 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:23.259 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:44:23.260 WARNING:rgw_multi.tests:bucket fwzbiz-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:44:28.262 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_version_suspended_incremental_sync ... FAIL 2020-05-24T14:44:28.263 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-21 2020-05-24T14:44:29.153 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-22 2020-05-24T14:44:31.078 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-23 2020-05-24T14:44:34.745 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:34.745 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:34.746 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:36.785 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:44:37.538 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590331474.143509_262.1","last_update":"2020-05-24 14:44:34.143509Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:44:37.538 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:44:37.539 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:37.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:37.540 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:40.384 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590331474.143509_262.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:44:34.143509Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:44:40.384 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:44:40.385 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:44:40.385 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:40.386 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:40.386 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:43.605 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590331474.143509_262.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:44:34.143509Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:44:43.606 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:44:43.606 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:44:43.607 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:43.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:43.608 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:46.656 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590331474.143509_262.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:44:34.143509Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:44:46.657 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:44:56.556 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:56.557 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:56.557 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:02.020 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000004.6.11"}]} 2020-05-24T14:45:02.021 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-21 2020-05-24T14:45:02.022 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:02.022 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:02.023 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:04.927 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"cj3HO53IxcgdJkgaJRkScJ-e3jeGGnz","ns":""},"count":2},"inc_marker":{"position":"0#00000000004.6.11"}}}] 2020-05-24T14:45:04.928 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-21 2020-05-24T14:45:04.929 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:04.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:04.930 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:07.411 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000004.6.11"}]} 2020-05-24T14:45:07.412 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-21 2020-05-24T14:45:07.413 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:07.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:07.414 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:09.961 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"cj3HO53IxcgdJkgaJRkScJ-e3jeGGnz","ns":""},"count":1},"inc_marker":{"position":"00000000004.6.11"}}}] 2020-05-24T14:45:09.962 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-21 2020-05-24T14:45:09.962 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:09.963 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:09.963 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:12.651 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000004.6.11"}]} 2020-05-24T14:45:12.651 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-21 2020-05-24T14:45:12.652 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:12.652 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:12.653 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:15.719 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"cj3HO53IxcgdJkgaJRkScJ-e3jeGGnz","ns":""},"count":2},"inc_marker":{"position":"0#00000000004.6.11"}}}] 2020-05-24T14:45:15.720 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-21 2020-05-24T14:45:15.720 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:15.721 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:15.721 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:18.158 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000003.6.5"}]} 2020-05-24T14:45:18.159 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-21 2020-05-24T14:45:18.159 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:18.160 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:18.160 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:21.389 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"cj3HO53IxcgdJkgaJRkScJ-e3jeGGnz","ns":""},"count":2},"inc_marker":{"position":"0#00000000003.6.5"}}}] 2020-05-24T14:45:21.389 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-21 2020-05-24T14:45:21.390 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:21.390 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:21.391 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:23.923 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000003.6.5"}]} 2020-05-24T14:45:23.924 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-21 2020-05-24T14:45:23.924 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:23.925 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:23.925 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:26.850 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"cj3HO53IxcgdJkgaJRkScJ-e3jeGGnz","ns":""},"count":2},"inc_marker":{"position":"0#00000000003.6.5"}}}] 2020-05-24T14:45:26.850 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-21 2020-05-24T14:45:26.851 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:26.852 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:26.852 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:29.408 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000003.6.5"}]} 2020-05-24T14:45:29.408 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-21 2020-05-24T14:45:29.409 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:29.409 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:29.410 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:32.407 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:32.408 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:45:37.409 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:37.410 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:37.411 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:40.240 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:40.240 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:45:45.241 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:45.242 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:45.243 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:48.398 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:48.399 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:45:53.400 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:53.401 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:53.402 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:56.411 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:56.412 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:01.414 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:01.415 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:01.415 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:04.601 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:04.614 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:09.615 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:09.616 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:09.617 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:12.851 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:12.852 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:17.854 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:17.854 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:17.855 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:21.057 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:21.057 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:26.058 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:26.059 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:26.060 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:29.623 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:29.623 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:34.625 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:34.626 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:34.627 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:38.154 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:38.155 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:43.157 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:43.158 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:43.159 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:46.273 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:46.274 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:51.276 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:51.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:51.277 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:54.393 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:54.394 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:46:59.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:59.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:59.396 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:02.816 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:02.816 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:47:07.818 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:07.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:07.819 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:11.030 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:11.031 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:47:15.150 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:58:25.368 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:15.167 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:58:25.387 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:15.586 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:58:25.805 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:16.032 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:16.033 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:16.034 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:19.087 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:19.088 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:47:24.090 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:24.091 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:24.092 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:27.401 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:27.402 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:47:32.404 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:32.405 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:32.405 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:35.493 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:35.493 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:47:40.495 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:40.496 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:40.497 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:43.577 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:43.578 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:47:48.579 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:48.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:48.581 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:51.727 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:51.728 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:47:52.506 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:59:02.725 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:52.509 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:59:02.728 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:52.578 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:00:59:02.798 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:56.729 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:56.730 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:56.730 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:59.377 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:59.378 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:48:04.379 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:04.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:04.380 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:07.426 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:07.426 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:48:12.427 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:12.428 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:12.429 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:15.492 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:15.493 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:48:20.494 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:20.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:20.496 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:23.578 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:23.579 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:48:28.581 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:28.582 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:28.583 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:31.583 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:31.584 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:48:35.429 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:58:38.614 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:35.438 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:58:38.623 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:35.750 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:58:38.935 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:36.585 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:36.586 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:36.587 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:39.577 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:39.578 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:48:44.579 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:44.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:44.581 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:47.635 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:47.635 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:48:52.637 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:52.638 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:52.638 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:55.646 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:55.647 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:00.647 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:00.648 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:00.649 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:03.715 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:59:06.900 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:49:03.740 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:59:06.926 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:49:03.771 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:03.771 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:03.782 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:00:59:06.967 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:49:08.773 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:08.774 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:08.775 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:11.835 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:11.836 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:16.838 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:16.838 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:16.839 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:19.817 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:19.818 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:24.819 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:24.821 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:24.821 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:28.051 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:28.052 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:33.053 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:33.054 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:33.055 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:35.810 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:35.811 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:40.812 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:40.813 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:40.814 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:43.865 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:43.866 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:48.867 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:48.868 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:48.869 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:51.836 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:51.837 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:49:56.838 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:56.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:56.840 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:59.935 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:59.936 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:50:04.942 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:04.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:04.944 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:08.358 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:08.359 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:50:13.360 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:13.361 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:13.361 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:16.349 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:16.350 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:50:21.351 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:21.352 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:21.353 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:24.606 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:24.607 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:50:29.611 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:29.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:29.613 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:32.945 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:32.946 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:50:37.948 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:37.949 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:37.949 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:41.089 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:41.091 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:50:46.093 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:46.094 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:46.095 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:49.071 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:49.071 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:50:54.073 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:54.074 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:54.075 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:57.081 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:57.082 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:02.083 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:02.084 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:02.085 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:05.330 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:05.331 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:10.333 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:10.334 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:10.334 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:13.538 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:13.539 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:18.541 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:18.542 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:18.543 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:21.536 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:21.537 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:26.538 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:26.540 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:26.541 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:29.751 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:29.752 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:34.753 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:34.754 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:34.755 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:37.591 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:37.591 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:42.593 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:42.594 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:42.596 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:45.711 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:45.712 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:50.714 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:50.715 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:50.715 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:54.200 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:54.201 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:51:59.203 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:59.204 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:59.205 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:02.431 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:02.432 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:52:07.434 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:07.435 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:07.435 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:10.409 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:10.410 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:52:15.412 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:15.413 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:15.413 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:18.527 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:18.528 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:52:23.530 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:23.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:23.531 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:26.521 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:26.522 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:52:31.523 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:31.524 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:31.525 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:34.797 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:34.797 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:52:39.799 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:39.800 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:39.801 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:42.875 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:42.876 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:52:47.877 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:47.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:47.879 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:50.801 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:50.801 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:52:55.803 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:55.804 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:55.804 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:59.140 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:59.141 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:53:04.142 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:04.143 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:04.144 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:07.275 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:53:07.275 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:53:12.277 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:12.278 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:12.278 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:15.350 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:53:15.350 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:53:20.351 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:20.352 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:20.353 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:23.714 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:53:23.715 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:53:28.716 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:28.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:28.718 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:32.103 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:53:32.104 WARNING:rgw_multi.tests:bucket fwzbiz-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.6.5 target= 2020-05-24T14:53:37.106 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_delete_marker_full_sync ... FAIL 2020-05-24T14:53:37.108 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-24 2020-05-24T14:53:37.891 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-25 2020-05-24T14:53:39.929 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-26 2020-05-24T14:53:45.195 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:45.196 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:45.197 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:47.221 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T14:53:47.861 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590332025.003642_307.1","last_update":"2020-05-24 14:53:45.003642Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T14:53:47.862 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:53:47.863 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:47.863 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:47.864 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:50.235 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332025.003642_307.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:53:45.003642Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:53:50.236 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:53:50.237 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:53:50.237 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:50.238 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:50.238 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:53.244 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332025.003642_307.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:53:45.003642Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:53:53.245 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:53:53.245 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:53:53.246 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:53.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:53.247 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:55.969 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332025.003642_307.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:53:45.003642Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:53:55.970 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:54:01.937 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:01.937 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:01.938 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:07.537 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000005.6.6"}]} 2020-05-24T14:54:07.538 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-24 2020-05-24T14:54:07.538 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:07.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:07.539 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:10.387 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000005.6.6"}}}] 2020-05-24T14:54:10.388 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-24 2020-05-24T14:54:10.389 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:10.389 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:10.390 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:10.657 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:05:20.876 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:54:10.684 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:05:20.903 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:54:11.086 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:05:21.305 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:54:14.038 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000005.6.6"}]} 2020-05-24T14:54:14.039 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-24 2020-05-24T14:54:14.039 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:14.040 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:14.040 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:17.091 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000005.6.6"}}}] 2020-05-24T14:54:17.092 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-24 2020-05-24T14:54:17.092 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:17.093 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:17.093 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:20.196 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000005.6.6"}]} 2020-05-24T14:54:20.197 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-24 2020-05-24T14:54:20.197 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:20.198 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:20.198 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:23.656 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000005.6.6"}}}] 2020-05-24T14:54:23.656 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-24 2020-05-24T14:54:23.657 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:23.658 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:23.658 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:26.074 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000001.4.12"}]} 2020-05-24T14:54:26.075 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-24 2020-05-24T14:54:26.075 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:26.076 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:26.076 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:28.882 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000001.4.12"}}}] 2020-05-24T14:54:28.883 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-24 2020-05-24T14:54:28.883 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:28.884 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:28.885 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000001.4.12"}]} 2020-05-24T14:54:31.608 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-24 2020-05-24T14:54:31.609 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:31.609 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:31.610 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:34.669 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000001.4.12"}}}] 2020-05-24T14:54:34.670 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-24 2020-05-24T14:54:34.670 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:34.671 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:34.671 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:37.230 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000001.4.12"}]} 2020-05-24T14:54:37.230 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-24 2020-05-24T14:54:37.231 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:37.232 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:37.232 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:40.447 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:40.447 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:54:45.449 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:45.450 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:45.451 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:48.433 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:48.434 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:54:53.435 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:53.436 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:53.436 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:56.758 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:56.759 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:01.761 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:01.762 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:01.762 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:04.841 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:04.841 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:09.842 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:09.843 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:09.844 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:13.454 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:13.455 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:18.457 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:18.458 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:18.458 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:21.488 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:21.489 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:26.490 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:26.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:26.492 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:29.485 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:29.486 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:29.770 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:05:32.955 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:55:29.786 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:05:32.971 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:55:30.089 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:05:33.273 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:55:34.487 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:34.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:34.488 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:37.891 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:37.892 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:42.893 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:42.893 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:42.894 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:45.917 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:45.918 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:50.919 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:50.920 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:50.921 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:54.354 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:54.354 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:55:59.356 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:59.357 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:59.357 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:02.743 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:02.744 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:56:07.746 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:07.746 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:07.747 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:11.038 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:11.039 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:56:13.221 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:07:23.440 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:56:13.246 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:07:23.465 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:56:13.320 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:07:23.540 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T14:56:16.041 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:16.042 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:16.043 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:20.062 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:20.063 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:56:25.065 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:25.066 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:25.067 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:29.587 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:29.588 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:56:34.589 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:34.590 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:34.591 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:38.553 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:38.554 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:56:43.555 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:43.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:43.557 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:46.794 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:46.795 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:56:51.796 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:51.797 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:51.798 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:54.826 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:54.827 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:56:59.828 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:59.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:59.830 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:03.455 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:03.456 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:57:08.457 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:08.458 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:08.459 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:11.427 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:11.428 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:57:16.431 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:16.432 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:16.433 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:19.639 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:19.640 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:57:24.474 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:07:27.659 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:57:24.477 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:07:27.662 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:57:24.560 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:07:27.745 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T14:57:24.642 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:24.643 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:24.643 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:27.942 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:27.943 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:57:32.944 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:32.945 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:32.946 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:36.122 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:36.123 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:57:41.124 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:41.125 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:41.125 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:43.951 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:43.952 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:57:48.953 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:48.954 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:48.954 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:51.956 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:51.956 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:57:56.958 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:56.958 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:56.959 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:00.005 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:00.006 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:58:05.007 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:05.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:05.009 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:08.445 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:08.446 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:58:13.447 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:13.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:13.450 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:16.538 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:16.539 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:58:21.542 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:21.543 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:21.543 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:25.095 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:25.096 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:58:30.098 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:30.098 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:30.099 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:33.462 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:33.463 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:58:38.464 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:38.465 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:38.466 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:41.363 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:41.532 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:58:46.534 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:46.535 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:46.536 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:49.755 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:49.756 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:58:54.758 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:54.759 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:54.760 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:57.989 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:57.989 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:59:02.991 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:02.992 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:02.992 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:06.215 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:06.216 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:59:11.217 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:11.218 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:11.219 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:14.933 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:14.934 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:59:19.936 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:19.937 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:19.939 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:23.740 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:23.741 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:59:28.743 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:28.744 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:28.746 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:31.827 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:31.828 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:59:36.829 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:36.830 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:36.831 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:40.018 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:40.019 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:59:45.021 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:45.021 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:45.022 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:47.998 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:47.998 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T14:59:53.000 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:53.001 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:53.001 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:56.411 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:56.412 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:01.414 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:01.415 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:01.416 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:04.986 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:04.987 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:09.988 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:09.989 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:09.990 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:13.239 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:13.240 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:18.242 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:18.243 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:18.243 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:21.393 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:21.394 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:26.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:26.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:26.397 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:29.885 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:29.886 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:34.887 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:34.888 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:34.889 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:38.099 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:38.099 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:43.101 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:43.102 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:43.103 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:46.295 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:46.296 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:51.298 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:51.299 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:51.300 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:54.497 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:54.498 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:00:59.499 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:59.500 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:59.501 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:02.751 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:02.751 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:01:06.156 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:12:16.374 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:01:06.160 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:12:16.380 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:01:06.382 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:12:16.601 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:01:07.754 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:07.755 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:07.756 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:11.061 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:11.128 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:01:16.130 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:16.131 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:16.131 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:19.715 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:19.716 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:01:24.717 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:24.718 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:24.719 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:28.006 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:28.006 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:01:33.008 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:33.009 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:33.009 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:36.401 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:36.402 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:01:41.403 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:41.404 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:41.405 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:44.665 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:44.666 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:01:49.667 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:49.669 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:49.669 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:52.891 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:01:57.894 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:57.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:57.896 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:01.358 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:01.359 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:02:06.360 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:06.361 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:06.362 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:09.656 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:09.657 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:02:14.658 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:14.659 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:14.660 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:17.822 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:17.822 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:02:22.824 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:22.825 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:22.826 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:26.080 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:12:29.264 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:02:26.084 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:12:29.268 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:02:26.177 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:26.178 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:02:26.302 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:12:29.486 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:02:31.179 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:31.180 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:31.180 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:34.710 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:34.711 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:02:39.712 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:39.713 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:39.713 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:43.117 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:43.117 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:02:48.119 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:48.120 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:48.120 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:51.133 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:51.134 WARNING:rgw_multi.tests:bucket fwzbiz-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.4.12 target= 2020-05-24T15:02:56.136 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_suspended_delete_marker_full_sync ... FAIL 2020-05-24T15:02:56.137 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-27 2020-05-24T15:02:57.010 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-28 2020-05-24T15:02:58.541 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-29 2020-05-24T15:03:02.834 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_versioning ... ok 2020-05-24T15:03:02.836 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-30 2020-05-24T15:03:04.074 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-31 2020-05-24T15:03:05.915 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-32 2020-05-24T15:03:10.018 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_acl ... ok 2020-05-24T15:03:10.019 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-33 2020-05-24T15:03:10.820 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-34 2020-05-24T15:03:12.931 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-35 2020-05-24T15:03:18.618 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_cors ... ok 2020-05-24T15:03:18.619 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-36 2020-05-24T15:03:19.558 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-37 2020-05-24T15:03:21.511 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-38 2020-05-24T15:03:22.843 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:22.844 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:22.845 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:25.028 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T15:03:25.632 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590332602.324881_396.1","last_update":"2020-05-24 15:03:22.324881Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:03:25.634 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:03:25.635 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:25.636 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:25.637 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:28.212 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332602.324881_396.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:22.324881Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:28.213 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:03:28.214 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:03:28.215 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:28.216 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:28.217 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:31.190 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332602.324881_396.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:22.324881Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:31.191 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:03:31.192 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:03:31.192 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:31.193 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:31.196 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:34.792 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332602.324881_396.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:22.324881Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:34.962 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:03:38.106 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_delete_notempty ... ok 2020-05-24T15:03:38.107 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-39 2020-05-24T15:03:41.500 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-40 2020-05-24T15:03:44.579 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-41 2020-05-24T15:03:45.995 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:45.996 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:45.997 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:48.150 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=1a5c7941-36e1-49b6-b10c-5aec63eb825b 2020-05-24T15:03:48.783 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590332625.395323_409.1","last_update":"2020-05-24 15:03:45.395323Z"},{"marker":"1_1590328801.559787_129.1","last_update":"2020-05-24 14:00:01.559787Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:03:48.784 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:03:48.785 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:48.786 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:48.786 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:51.334 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332625.395323_409.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:45.395323Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:51.335 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:03:51.335 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:03:51.336 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:51.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:51.337 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:54.129 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332625.395323_409.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:45.395323Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:54.129 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:03:54.130 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:03:54.130 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:54.131 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:54.131 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:57.207 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332625.395323_409.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:45.395323Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:57.207 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:03:57.208 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2020-05-24T15:03:57.209 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T15:03:57.255 INFO:tasks.rgw.c1.client.1.smithi019.stdout:2020-05-24 15:03:57.213 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 16107) UID: 0 2020-05-24T15:03:57.262 INFO:tasks.rgw.c1.client.1.smithi019.stdout:2020-05-24 15:03:57.222 4049040 -1 shutting down 2020-05-24T15:04:03.318 INFO:tasks.rgw.c1.client.1:Stopped 2020-05-24T15:04:03.319 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:03.320 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:03.320 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:04:03.841 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:04:03.839 7f694c617840 0 NOTICE: overriding master zone: 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T15:04:03.855 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:04:03.852 7f694c617840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T15:04:03.856 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:04:03.853 7f694c617840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T15:04:03.898 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:04:03.895 7f694c617840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T15:04:03.898 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:04:03.895 7f694c617840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T15:04:04.011 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:04:04.012 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:04.012 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:04.013 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:04:08.659 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"a91bfc36-ad1f-4fbc-8718-897c542ad285","epoch":1,"predecessor_uuid":"1a5c7941-36e1-49b6-b10c-5aec63eb825b","sync_status":["1_1590332625.395323_409.1","1_1590328801.559787_129.1","",""],"period_map":{"id":"a91bfc36-ad1f-4fbc-8718-897c542ad285","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"692d187d-2dfb-4f37-a732-96d22188c425","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"692d187d-2dfb-4f37-a732-96d22188c425","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":3} 2020-05-24T15:04:08.660 INFO:rgw_multi.tests:Set master zone=test-zone2, waiting 60s for reconfiguration.. 2020-05-24T15:04:34.009 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:15:44.227 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:04:34.014 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:15:44.233 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:04:34.118 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:15:44.337 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:05:08.661 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-42 2020-05-24T15:05:10.146 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-43 2020-05-24T15:05:11.112 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:05:11.113 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:05:11.113 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:05:12.985 INFO:teuthology.orchestra.run.smithi193.stderr:No --period given, using current period=a91bfc36-ad1f-4fbc-8718-897c542ad285 2020-05-24T15:05:13.576 INFO:teuthology.orchestra.run.smithi193.stdout:[{"marker":"1_1590332710.970452_141.1","last_update":"2020-05-24 15:05:10.970452Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:05:13.577 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone1 2020-05-24T15:05:13.578 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:05:13.578 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:05:13.579 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:05:16.426 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"a91bfc36-ad1f-4fbc-8718-897c542ad285","realm_epoch":3},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332710.970452_141.1","next_step_marker":"","total_entries":18,"pos":0,"timestamp":"2020-05-24 15:05:10.970452Z","realm_epoch":3}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":47,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":14,"pos":0,"timestamp":"0.000000","realm_epoch":3}}]},"full_sync":{"total":80,"complete":80}} 2020-05-24T15:05:16.426 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone1 2020-05-24T15:05:16.427 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:05:16.427 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:05:16.428 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone modify --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:05:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:05:16.859 7fcfb27c7840 0 NOTICE: overriding master zone: 692d187d-2dfb-4f37-a732-96d22188c425 2020-05-24T15:05:16.938 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:05:16.934 7fcfb27c7840 0 failed reading obj info from .rgw.root:zone_info.692d187d-2dfb-4f37-a732-96d22188c425: (2) No such file or directory 2020-05-24T15:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:05:16.934 7fcfb27c7840 0 WARNING: could not read zone params for zone id=692d187d-2dfb-4f37-a732-96d22188c425 name=test-zone2 2020-05-24T15:05:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:05:16.946 7fcfb27c7840 0 failed reading obj info from .rgw.root:zone_info.7f588a25-0d83-455c-8d94-e1a778babeb4: (2) No such file or directory 2020-05-24T15:05:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:05:16.946 7fcfb27c7840 0 WARNING: could not read zone params for zone id=7f588a25-0d83-455c-8d94-e1a778babeb4 name=test-zone4 2020-05-24T15:05:17.082 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:05:17.083 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:05:17.083 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:05:17.084 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:05:21.856 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":1,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:05:21.857 INFO:rgw_multi.tests:Set master zone=test-zone1, waiting 60s for reconfiguration.. 2020-05-24T15:05:45.119 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:15:48.302 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:05:45.138 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:15:48.322 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:05:45.239 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:15:48.424 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:06:21.859 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-44 2020-05-24T15:06:22.861 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-45 2020-05-24T15:06:24.430 INFO:tasks.rgw.c1.client.1:Restarting daemon 2020-05-24T15:06:24.451 INFO:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2020-05-24T15:06:24.456 INFO:tasks.rgw.c1.client.1:Started 2020-05-24T15:06:24.457 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T15:06:24.531 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:24.531 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:24.534 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:06:24.535 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:06:25.536 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T15:06:25.589 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:25.589 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:25.595 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:06:25.596 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:06:27.597 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T15:06:27.679 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:27.680 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:27.683 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:06:27.684 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:06:31.686 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T15:06:31.753 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:31.754 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:31.757 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi019.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:06:31.758 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:06:39.759 INFO:teuthology.orchestra.run.smithi019:> curl http://smithi019.front.sepia.ceph.com:8001 2020-05-24T15:06:39.808 INFO:teuthology.orchestra.run.smithi019.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:39.808 INFO:teuthology.orchestra.run.smithi019.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:39.951 INFO:teuthology.orchestra.run.smithi019.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 1494 0 --:--:-- --:--:-- --:--:-- 1496 2020-05-24T15:06:39.952 INFO:teuthology.orchestra.run.smithi019.stdout:anonymous 2020-05-24T15:06:39.952 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:39.953 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:39.954 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=9420dac7-f825-4d7d-a354-f516953dfe80 2020-05-24T15:06:43.950 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590332783.852667_96.1","last_update":"2020-05-24 15:06:23.852667Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:06:43.951 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:06:43.952 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:43.952 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:43.953 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:46.500 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332783.852667_96.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:06:23.852667Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:06:46.501 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:06:46.502 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:06:46.502 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:46.503 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:46.503 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:49.609 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332783.852667_96.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:06:23.852667Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:06:49.610 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:06:49.611 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:06:49.611 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:49.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:49.613 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:52.872 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332783.852667_96.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:06:23.852667Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:06:52.872 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:06:52.873 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-39 zones={test-zone2, test-zone1} 2020-05-24T15:06:53.408 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:53.587 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:53.835 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-39 zones={test-zone2, test-zone1} 2020-05-24T15:06:53.835 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-39 zones={test-zone3, test-zone1} 2020-05-24T15:06:54.270 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:54.419 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:54.612 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-39 zones={test-zone3, test-zone1} 2020-05-24T15:06:54.613 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-39 zones={test-zone1, test-zone2} 2020-05-24T15:06:54.797 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:54.848 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:55.070 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-39 zones={test-zone1, test-zone2} 2020-05-24T15:06:55.071 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-39 zones={test-zone3, test-zone2} 2020-05-24T15:06:55.190 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:55.239 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:55.448 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-39 zones={test-zone3, test-zone2} 2020-05-24T15:06:55.449 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-39 zones={test-zone1, test-zone3} 2020-05-24T15:06:55.542 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:55.602 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:55.763 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-39 zones={test-zone1, test-zone3} 2020-05-24T15:06:55.763 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-39 zones={test-zone2, test-zone3} 2020-05-24T15:06:55.927 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:55.996 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:56.152 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-39 zones={test-zone2, test-zone3} 2020-05-24T15:06:56.153 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-40 zones={test-zone2, test-zone1} 2020-05-24T15:06:56.509 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:56.677 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:56.914 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-40 zones={test-zone2, test-zone1} 2020-05-24T15:06:56.915 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-40 zones={test-zone3, test-zone1} 2020-05-24T15:06:57.322 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:57.417 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:57.646 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-40 zones={test-zone3, test-zone1} 2020-05-24T15:06:57.647 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-40 zones={test-zone1, test-zone2} 2020-05-24T15:06:57.804 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:57.878 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:58.115 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-40 zones={test-zone1, test-zone2} 2020-05-24T15:06:58.116 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-40 zones={test-zone3, test-zone2} 2020-05-24T15:06:58.258 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:58.317 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:58.573 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-40 zones={test-zone3, test-zone2} 2020-05-24T15:06:58.578 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-40 zones={test-zone1, test-zone3} 2020-05-24T15:06:58.704 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:58.791 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:59.022 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-40 zones={test-zone1, test-zone3} 2020-05-24T15:06:59.023 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-40 zones={test-zone2, test-zone3} 2020-05-24T15:06:59.190 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:59.306 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:59.484 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-40 zones={test-zone2, test-zone3} 2020-05-24T15:06:59.485 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-41 zones={test-zone2, test-zone1} 2020-05-24T15:06:59.920 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:00.039 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:00.240 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-41 zones={test-zone2, test-zone1} 2020-05-24T15:07:00.241 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-41 zones={test-zone3, test-zone1} 2020-05-24T15:07:00.564 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:00.662 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:00.901 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-41 zones={test-zone3, test-zone1} 2020-05-24T15:07:00.902 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-41 zones={test-zone1, test-zone2} 2020-05-24T15:07:01.113 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:01.174 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:01.361 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-41 zones={test-zone1, test-zone2} 2020-05-24T15:07:01.362 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-41 zones={test-zone3, test-zone2} 2020-05-24T15:07:01.491 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:01.558 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:01.754 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-41 zones={test-zone3, test-zone2} 2020-05-24T15:07:01.755 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-41 zones={test-zone1, test-zone3} 2020-05-24T15:07:01.882 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:01.951 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:02.143 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-41 zones={test-zone1, test-zone3} 2020-05-24T15:07:02.144 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-41 zones={test-zone2, test-zone3} 2020-05-24T15:07:02.291 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:02.352 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:02.507 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-41 zones={test-zone2, test-zone3} 2020-05-24T15:07:02.508 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-42 zones={test-zone2, test-zone1} 2020-05-24T15:07:02.626 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:02.681 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:02.851 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-42 zones={test-zone2, test-zone1} 2020-05-24T15:07:02.852 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-42 zones={test-zone3, test-zone1} 2020-05-24T15:07:02.975 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:03.056 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:03.288 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-42 zones={test-zone3, test-zone1} 2020-05-24T15:07:03.289 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-42 zones={test-zone1, test-zone2} 2020-05-24T15:07:03.459 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:03.519 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:03.769 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-42 zones={test-zone1, test-zone2} 2020-05-24T15:07:03.770 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-42 zones={test-zone3, test-zone2} 2020-05-24T15:07:03.890 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:03.938 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:04.128 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-42 zones={test-zone3, test-zone2} 2020-05-24T15:07:04.128 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-42 zones={test-zone1, test-zone3} 2020-05-24T15:07:04.281 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:04.352 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:04.514 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-42 zones={test-zone1, test-zone3} 2020-05-24T15:07:04.515 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-42 zones={test-zone2, test-zone3} 2020-05-24T15:07:04.706 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:04.784 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:05.031 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-42 zones={test-zone2, test-zone3} 2020-05-24T15:07:05.031 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-43 zones={test-zone2, test-zone1} 2020-05-24T15:07:05.270 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:05.406 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:05.873 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-43 zones={test-zone2, test-zone1} 2020-05-24T15:07:05.874 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-43 zones={test-zone3, test-zone1} 2020-05-24T15:07:06.134 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:06.234 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:06.445 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-43 zones={test-zone3, test-zone1} 2020-05-24T15:07:06.446 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-43 zones={test-zone1, test-zone2} 2020-05-24T15:07:06.616 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:06.710 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:06.945 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-43 zones={test-zone1, test-zone2} 2020-05-24T15:07:06.946 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-43 zones={test-zone3, test-zone2} 2020-05-24T15:07:07.043 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:07.157 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:07.419 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-43 zones={test-zone3, test-zone2} 2020-05-24T15:07:07.420 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-43 zones={test-zone1, test-zone3} 2020-05-24T15:07:07.552 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:07.627 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:07.818 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-43 zones={test-zone1, test-zone3} 2020-05-24T15:07:07.818 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-43 zones={test-zone2, test-zone3} 2020-05-24T15:07:08.013 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:08.099 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:08.328 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-43 zones={test-zone2, test-zone3} 2020-05-24T15:07:08.329 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-44 zones={test-zone2, test-zone1} 2020-05-24T15:07:08.481 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:08.549 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:08.792 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-44 zones={test-zone2, test-zone1} 2020-05-24T15:07:08.793 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-44 zones={test-zone3, test-zone1} 2020-05-24T15:07:08.913 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:08.975 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:09.138 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-44 zones={test-zone3, test-zone1} 2020-05-24T15:07:09.139 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-44 zones={test-zone1, test-zone2} 2020-05-24T15:07:09.331 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:09.446 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:09.710 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-44 zones={test-zone1, test-zone2} 2020-05-24T15:07:09.711 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-44 zones={test-zone3, test-zone2} 2020-05-24T15:07:09.884 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:09.981 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:10.244 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-44 zones={test-zone3, test-zone2} 2020-05-24T15:07:10.245 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-44 zones={test-zone1, test-zone3} 2020-05-24T15:07:10.385 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:10.440 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:10.625 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-44 zones={test-zone1, test-zone3} 2020-05-24T15:07:10.626 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-44 zones={test-zone2, test-zone3} 2020-05-24T15:07:10.774 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:10.840 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:11.057 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-44 zones={test-zone2, test-zone3} 2020-05-24T15:07:11.057 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-45 zones={test-zone2, test-zone1} 2020-05-24T15:07:11.220 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:11.336 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:11.582 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-45 zones={test-zone2, test-zone1} 2020-05-24T15:07:11.583 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-45 zones={test-zone3, test-zone1} 2020-05-24T15:07:11.771 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:11.829 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:12.016 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-45 zones={test-zone3, test-zone1} 2020-05-24T15:07:12.016 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-45 zones={test-zone1, test-zone2} 2020-05-24T15:07:12.197 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:12.265 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:12.490 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-45 zones={test-zone1, test-zone2} 2020-05-24T15:07:12.491 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-45 zones={test-zone3, test-zone2} 2020-05-24T15:07:12.627 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:12.694 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:12.912 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-45 zones={test-zone3, test-zone2} 2020-05-24T15:07:12.913 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-45 zones={test-zone1, test-zone3} 2020-05-24T15:07:13.078 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:13.141 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:13.312 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-45 zones={test-zone1, test-zone3} 2020-05-24T15:07:13.313 INFO:tasks.rgw_multi.zone_rados:comparing bucket=fwzbiz-45 zones={test-zone2, test-zone3} 2020-05-24T15:07:13.492 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:07:13.583 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:07:13.831 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=fwzbiz-45 zones={test-zone2, test-zone3} 2020-05-24T15:07:13.832 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:13.832 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:13.833 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:17.257 INFO:teuthology.orchestra.run.smithi019.stdout:[{"id":"1_1590328592.508687_53.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.508687920Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328592.917777_55.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.917777969Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328813.921181_64.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:13.921181724Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.103446_65.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:14.103446504Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.184865_66.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.184865631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.400239_67.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.400239762Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.850168_69.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.850168976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.993137_70.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.993137458Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328815.062089_71.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.062089519Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"write"}}},{"id":"1_1590328815.386281_72.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.386281164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.719132_73.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.719132889Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328816.893052_74.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.893052032Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.948640_75.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:16.948640933Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590328817.073639_76.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:17.073639954Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.402320_96.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.402320164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.697935_97.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.697935376Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.752878_98.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.752878712Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.901862_99.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.901862037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.347957_101.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.347957633Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.486510_102.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.486510226Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.577678_103.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.577678314Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.842804_104.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.842804643Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.251362_105.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.251362523Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.399140_106.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.399140495Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.482971_107.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.482971589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.625971_108.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.625971369Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.665732_154.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.665732383Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"write"}}},{"id":"1_1590329883.848009_155.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.848009328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.936799_156.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:03.936799083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"write"}}},{"id":"1_1590329884.074924_157.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:04.074924748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329884.735428_158.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:04.735428932Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.052055_159.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:05.052055284Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329885.210786_160.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.210786752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.548749_161.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.548749753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.537795_162.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.537795828Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"write"}}},{"id":"1_1590329886.669790_163.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.669790405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.726948_164.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:06.726948984Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"write"}}},{"id":"1_1590329887.030128_165.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:07.030128239Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329887.823525_166.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:07.823525444Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.097199_167.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:08.097199890Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329888.329299_168.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.329299790Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.521426_169.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.521426544Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329889.036851_170.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.036851814Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"write"}}},{"id":"1_1590329889.537905_171.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.537905497Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330412.885284_207.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:52.885284053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.167362_208.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.167362293Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.256206_209.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.256206617Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.402854_210.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.402854007Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.582718_211.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.582718645Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"write"}}},{"id":"1_1590330413.778058_212.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.778058231Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330936.244874_222.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.244874907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.530565_223.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.530565189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330936.597653_224.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.597653809Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.762225_225.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.762225748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.505378_245.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.505378326Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"write"}}},{"id":"1_1590331468.710798_246.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.710798130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.843043_247.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:28.843043489Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590331469.009382_248.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:29.009382468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331469.800887_249.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:29.800887834Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.193693_250.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:30.193693841Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331470.301973_251.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.301973037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.455201_252.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.455201532Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331471.584345_253.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.584345083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"write"}}},{"id":"1_1590331471.850957_254.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.850957150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.003811_255.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.003811342Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"write"}}},{"id":"1_1590331472.200489_256.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.200489331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.703779_257.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:32.703779771Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.055080_258.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:33.055080161Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.401514_259.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.401514029Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.624466_260.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.624466894Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.927577_261.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:33.927577797Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"write"}}},{"id":"1_1590331474.143509_262.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:34.143509836Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332017.477105_282.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.477105632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.591917_283.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.591917171Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332017.652491_284.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.652491494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.767809_285.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.767809971Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332018.626827_286.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.626827980Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332018.983994_287.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.983994356Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332019.048026_288.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.048026105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"write"}}},{"id":"1_1590332019.311787_289.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.311787349Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.450116_290.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.450116581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.603092_291.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.603092627Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.702109_292.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.702109684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.816758_293.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.816758240Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332021.409529_295.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.409529984Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332021.814058_296.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.814058013Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332021.994940_297.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.994940878Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"write"}}},{"id":"1_1590332022.279636_298.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:42.279636992Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332022.525693_299.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.525693290Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"write"}}},{"id":"1_1590332022.817665_300.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.817665067Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332023.133730_301.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.133730721Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"write"}}},{"id":"1_1590332023.446055_302.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.446055762Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332024.079336_303.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.079336921Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332024.395330_304.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.395330949Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332024.698623_306.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.698623067Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"write"}}},{"id":"1_1590332025.003642_307.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:45.003642510Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332576.308624_331.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.308624985Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.530859_332.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.530859626Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332576.609981_333.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.609981019Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.781799_334.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.781799684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.537460_335.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.537460251Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.728195_336.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.728195704Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.788398_337.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.788398528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.960484_338.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.960484471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.162679_339.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.162679805Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.377176_340.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.377176705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.440183_341.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.440183470Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.573394_342.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.573394776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332580.441736_343.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.441736738Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"write"}}},{"id":"1_1590332580.635748_344.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.635748106Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332581.090074_345.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.090074825Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"write"}}},{"id":"1_1590332581.758248_346.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.758248152Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332582.184585_347.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.184585859Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"write"}}},{"id":"1_1590332582.479754_348.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.479754685Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332583.194378_349.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.194378582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.462964_350.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.462964736Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332583.578177_351.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.578177872Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.730991_352.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.730991086Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332584.691763_353.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.691763614Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332584.947121_354.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.947121755Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332585.103707_355.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.103707310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332585.232605_356.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.232605182Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.344013_357.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.344013365Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.464539_358.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.464539687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.582866_359.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.582866987Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.748023_360.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.748023907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332587.677153_361.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.677153663Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"write"}}},{"id":"1_1590332587.932544_362.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.932544635Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332588.327537_363.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.327537080Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"write"}}},{"id":"1_1590332588.619061_364.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.619061527Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332589.216798_365.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.216798093Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"write"}}},{"id":"1_1590332589.561035_366.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.561035178Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332590.255025_367.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.255025912Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.446227_368.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.446227608Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332590.502559_369.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.502559078Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.649027_370.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.649027123Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.375800_371.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.375800124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332591.742382_372.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.742382237Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.842552_373.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:11.842552021Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.031908_374.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:12.031908659Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.240469_375.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.240469031Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.391002_376.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.391002471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.465847_377.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.465847570Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.685001_378.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.685001719Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332594.450933_379.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.450933305Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"write"}}},{"id":"1_1590332594.858645_380.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.858645158Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332596.126321_381.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.126321558Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"write"}}},{"id":"1_1590332596.379140_382.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.379140830Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332597.929328_383.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:17.929328464Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"write"}}},{"id":"1_1590332598.267606_384.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:18.267606593Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332598.971437_385.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:18.971437370Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.187031_386.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:19.187031446Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332599.250412_387.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.250412420Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.365739_388.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.365739368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.095396_389.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.095396102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.271093_390.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.271093243Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.332512_391.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.332512615Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.459857_392.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.459857112Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332601.952044_393.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:21.952044861Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.098949_394.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:22.098949881Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332602.200693_395.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.200693657Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.324881_396.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.324881490Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332618.489788_398.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:38.489788797Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332619.055324_399.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:39.055324511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332619.292390_400.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:39.292390413Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332620.534454_401.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:40.534454583Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332622.462827_402.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:42.462827592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.169124_403.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:43.169124859Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332623.408767_404.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.408767528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.748848_405.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.748848044Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332624.895711_406.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:44.895711105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.057684_407.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:45.057684752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332625.146752_408.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.146752921Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.395323_409.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.395323934Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328687.177911_34.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:07.177911653Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328692.069642_35.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:12.069642200Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328692.266320_36.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:12.266320581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328693.128004_37.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:13.128004764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.151011_38.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:15.151011218Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.949640_39.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:17.949640848Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328700.795017_40.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:20.795017810Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328702.912912_41.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:22.912912206Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.578055_42.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.578055891Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.729955_43.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.729955016Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.799034_44.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.799034852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.981332_45.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.981332310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.253514_46.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.253514189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.389746_47.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.389746469Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.462701_48.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.462701880Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.646948_49.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.646948705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.108322_50.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.108322825Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.278473_51.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.278473753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.347900_52.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.347900543Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.546580_53.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.546580127Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328742.949663_54.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:02.949663616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.100429_55.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:03.100429758Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328743.339199_56.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.339199274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.504813_57.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.504813479Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.417840_58.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.417840348Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.529097_59.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.529097974Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.586991_60.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.586991054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.713299_61.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.713299225Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328756.828867_62.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.828867828Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328757.071284_63.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:17.071284402Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328757.495644_64.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.495644501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.636125_65.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.636125563Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328757.719832_66.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.719832184Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.845993_67.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.845993517Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328758.093585_68.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.093585822Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328758.317456_69.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.317456261Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328758.691521_70.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.691521521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328758.854008_71.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.854008793Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328758.982217_72.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:18.982217287Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"write"}}},{"id":"1_1590328759.162486_73.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:19.162486093Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328759.283606_74.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.283606303Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328759.467567_75.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.467567593Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328760.133426_76.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.133426441Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.326796_77.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.326796578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328760.396609_78.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.396609688Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.501441_79.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.501441090Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328760.605321_80.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.605321185Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328760.874531_81.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.874531368Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328761.934150_82.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:21.934150091Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.135705_83.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.135705114Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328762.220642_84.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.220642897Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.351162_85.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.351162800Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328762.451877_86.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.451877182Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328762.616596_87.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.616596068Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328763.544481_88.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.544481776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"write"}}},{"id":"1_1590328763.780813_89.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.780813494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328763.875460_90.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:23.875460851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"write"}}},{"id":"1_1590328764.115279_91.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:24.115279014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328764.302206_92.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.302206541Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.586734_93.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.586734819Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.603699_94.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.603699796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328765.804758_95.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.804758399Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328765.871314_96.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:25.871314468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"write"}}},{"id":"1_1590328766.001613_97.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:26.001613252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328766.178702_98.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.178702867Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328766.369723_99.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.369723003Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.147013_100.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.147013040Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.356936_101.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.356936965Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328767.419119_102.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.419119272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.609379_103.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.609379045Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328767.717880_104.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.717880724Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.877492_105.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.877492811Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328768.879650_106.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:28.879650152Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.001689_107.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.001689187Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328769.060092_108.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.060092214Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.162808_109.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.162808526Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328769.343071_110.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.343071979Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328769.600141_111.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.600141569Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328783.361580_112.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.361580865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.503670_113.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.503670361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.576366_114.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.576366221Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.715625_115.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.715625577Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.472869_116.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.472869224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.742386_117.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.742386838Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.812919_118.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.812919362Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.979340_119.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.979340935Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328785.869729_120.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:45.869729749Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.066327_121.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:46.066327601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328786.205110_122.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.205110843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.554014_123.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.554014976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328798.827667_124.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:58.827667322Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.003141_125.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:59.003141030Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328799.697584_126.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.697584863Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.894092_127.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.894092024Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328801.238053_128.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.238053772Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328801.559787_129.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.559787262Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:07:17.261 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:17.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:17.262 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:20.334 INFO:teuthology.orchestra.run.smithi193.stdout:[{"id":"1_1590328592.508687_53.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.508687920Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328592.917777_55.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.917777969Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328813.921181_64.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:13.921181724Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.103446_65.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:14.103446504Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.184865_66.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.184865631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.400239_67.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.400239762Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.850168_69.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.850168976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.993137_70.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.993137458Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328815.062089_71.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.062089519Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"write"}}},{"id":"1_1590328815.386281_72.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.386281164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.719132_73.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.719132889Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328816.893052_74.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.893052032Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.948640_75.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:16.948640933Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590328817.073639_76.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:17.073639954Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.402320_96.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.402320164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.697935_97.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.697935376Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.752878_98.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.752878712Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.901862_99.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.901862037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.347957_101.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.347957633Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.486510_102.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.486510226Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.577678_103.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.577678314Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.842804_104.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.842804643Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.251362_105.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.251362523Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.399140_106.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.399140495Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.482971_107.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.482971589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.625971_108.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.625971369Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.665732_154.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.665732383Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"write"}}},{"id":"1_1590329883.848009_155.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.848009328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.936799_156.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:03.936799083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"write"}}},{"id":"1_1590329884.074924_157.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:04.074924748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329884.735428_158.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:04.735428932Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.052055_159.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:05.052055284Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329885.210786_160.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.210786752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.548749_161.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.548749753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.537795_162.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.537795828Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"write"}}},{"id":"1_1590329886.669790_163.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.669790405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.726948_164.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:06.726948984Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"write"}}},{"id":"1_1590329887.030128_165.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:07.030128239Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329887.823525_166.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:07.823525444Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.097199_167.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:08.097199890Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329888.329299_168.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.329299790Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.521426_169.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.521426544Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329889.036851_170.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.036851814Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"write"}}},{"id":"1_1590329889.537905_171.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.537905497Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330412.885284_207.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:52.885284053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.167362_208.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.167362293Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.256206_209.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.256206617Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.402854_210.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.402854007Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.582718_211.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.582718645Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"write"}}},{"id":"1_1590330413.778058_212.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.778058231Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330936.244874_222.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.244874907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.530565_223.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.530565189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330936.597653_224.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.597653809Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.762225_225.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.762225748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.505378_245.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.505378326Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"write"}}},{"id":"1_1590331468.710798_246.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.710798130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.843043_247.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:28.843043489Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590331469.009382_248.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:29.009382468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331469.800887_249.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:29.800887834Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.193693_250.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:30.193693841Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331470.301973_251.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.301973037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.455201_252.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.455201532Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331471.584345_253.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.584345083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"write"}}},{"id":"1_1590331471.850957_254.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.850957150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.003811_255.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.003811342Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"write"}}},{"id":"1_1590331472.200489_256.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.200489331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.703779_257.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:32.703779771Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.055080_258.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:33.055080161Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.401514_259.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.401514029Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.624466_260.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.624466894Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.927577_261.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:33.927577797Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"write"}}},{"id":"1_1590331474.143509_262.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:34.143509836Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332017.477105_282.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.477105632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.591917_283.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.591917171Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332017.652491_284.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.652491494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.767809_285.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.767809971Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332018.626827_286.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.626827980Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332018.983994_287.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.983994356Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332019.048026_288.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.048026105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"write"}}},{"id":"1_1590332019.311787_289.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.311787349Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.450116_290.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.450116581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.603092_291.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.603092627Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.702109_292.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.702109684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.816758_293.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.816758240Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332021.409529_295.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.409529984Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332021.814058_296.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.814058013Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332021.994940_297.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.994940878Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"write"}}},{"id":"1_1590332022.279636_298.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:42.279636992Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332022.525693_299.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.525693290Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"write"}}},{"id":"1_1590332022.817665_300.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.817665067Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332023.133730_301.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.133730721Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"write"}}},{"id":"1_1590332023.446055_302.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.446055762Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332024.079336_303.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.079336921Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332024.395330_304.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.395330949Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332024.698623_306.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.698623067Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"write"}}},{"id":"1_1590332025.003642_307.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:45.003642510Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332576.308624_331.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.308624985Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.530859_332.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.530859626Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332576.609981_333.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.609981019Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.781799_334.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.781799684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.537460_335.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.537460251Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.728195_336.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.728195704Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.788398_337.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.788398528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.960484_338.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.960484471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.162679_339.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.162679805Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.377176_340.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.377176705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.440183_341.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.440183470Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.573394_342.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.573394776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332580.441736_343.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.441736738Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"write"}}},{"id":"1_1590332580.635748_344.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.635748106Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332581.090074_345.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.090074825Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"write"}}},{"id":"1_1590332581.758248_346.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.758248152Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332582.184585_347.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.184585859Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"write"}}},{"id":"1_1590332582.479754_348.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.479754685Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332583.194378_349.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.194378582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.462964_350.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.462964736Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332583.578177_351.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.578177872Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.730991_352.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.730991086Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332584.691763_353.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.691763614Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332584.947121_354.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.947121755Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332585.103707_355.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.103707310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332585.232605_356.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.232605182Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.344013_357.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.344013365Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.464539_358.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.464539687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.582866_359.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.582866987Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.748023_360.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.748023907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332587.677153_361.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.677153663Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"write"}}},{"id":"1_1590332587.932544_362.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.932544635Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332588.327537_363.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.327537080Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"write"}}},{"id":"1_1590332588.619061_364.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.619061527Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332589.216798_365.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.216798093Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"write"}}},{"id":"1_1590332589.561035_366.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.561035178Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332590.255025_367.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.255025912Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.446227_368.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.446227608Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332590.502559_369.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.502559078Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.649027_370.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.649027123Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.375800_371.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.375800124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332591.742382_372.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.742382237Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.842552_373.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:11.842552021Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.031908_374.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:12.031908659Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.240469_375.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.240469031Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.391002_376.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.391002471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.465847_377.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.465847570Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.685001_378.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.685001719Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332594.450933_379.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.450933305Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"write"}}},{"id":"1_1590332594.858645_380.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.858645158Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332596.126321_381.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.126321558Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"write"}}},{"id":"1_1590332596.379140_382.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.379140830Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332597.929328_383.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:17.929328464Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"write"}}},{"id":"1_1590332598.267606_384.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:18.267606593Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332598.971437_385.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:18.971437370Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.187031_386.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:19.187031446Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332599.250412_387.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.250412420Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.365739_388.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.365739368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.095396_389.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.095396102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.271093_390.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.271093243Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.332512_391.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.332512615Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.459857_392.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.459857112Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332601.952044_393.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:21.952044861Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.098949_394.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:22.098949881Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332602.200693_395.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.200693657Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.324881_396.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.324881490Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332618.489788_398.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:38.489788797Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332619.055324_399.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:39.055324511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332619.292390_400.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:39.292390413Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332620.534454_401.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:40.534454583Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332622.462827_402.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:42.462827592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.169124_403.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:43.169124859Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332623.408767_404.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.408767528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.748848_405.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.748848044Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332624.895711_406.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:44.895711105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.057684_407.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:45.057684752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332625.146752_408.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.146752921Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.395323_409.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.395323934Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328687.177911_34.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:07.177911653Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328692.069642_35.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:12.069642200Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328692.266320_36.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:12.266320581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328693.128004_37.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:13.128004764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.151011_38.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:15.151011218Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.949640_39.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:17.949640848Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328700.795017_40.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:20.795017810Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328702.912912_41.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:22.912912206Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.578055_42.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.578055891Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.729955_43.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.729955016Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.799034_44.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.799034852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.981332_45.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.981332310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.253514_46.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.253514189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.389746_47.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.389746469Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.462701_48.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.462701880Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.646948_49.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.646948705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.108322_50.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.108322825Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.278473_51.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.278473753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.347900_52.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.347900543Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.546580_53.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.546580127Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328742.949663_54.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:02.949663616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.100429_55.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:03.100429758Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328743.339199_56.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.339199274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.504813_57.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.504813479Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.417840_58.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.417840348Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.529097_59.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.529097974Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.586991_60.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.586991054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.713299_61.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.713299225Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328756.828867_62.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.828867828Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328757.071284_63.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:17.071284402Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328757.495644_64.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.495644501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.636125_65.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.636125563Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328757.719832_66.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.719832184Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.845993_67.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.845993517Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328758.093585_68.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.093585822Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328758.317456_69.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.317456261Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328758.691521_70.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.691521521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328758.854008_71.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.854008793Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328758.982217_72.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:18.982217287Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"write"}}},{"id":"1_1590328759.162486_73.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:19.162486093Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328759.283606_74.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.283606303Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328759.467567_75.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.467567593Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328760.133426_76.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.133426441Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.326796_77.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.326796578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328760.396609_78.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.396609688Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.501441_79.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.501441090Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328760.605321_80.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.605321185Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328760.874531_81.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.874531368Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328761.934150_82.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:21.934150091Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.135705_83.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.135705114Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328762.220642_84.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.220642897Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.351162_85.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.351162800Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328762.451877_86.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.451877182Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328762.616596_87.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.616596068Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328763.544481_88.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.544481776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"write"}}},{"id":"1_1590328763.780813_89.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.780813494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328763.875460_90.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:23.875460851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"write"}}},{"id":"1_1590328764.115279_91.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:24.115279014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328764.302206_92.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.302206541Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.586734_93.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.586734819Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.603699_94.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.603699796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328765.804758_95.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.804758399Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328765.871314_96.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:25.871314468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"write"}}},{"id":"1_1590328766.001613_97.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:26.001613252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328766.178702_98.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.178702867Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328766.369723_99.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.369723003Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.147013_100.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.147013040Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.356936_101.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.356936965Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328767.419119_102.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.419119272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.609379_103.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.609379045Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328767.717880_104.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.717880724Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.877492_105.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.877492811Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328768.879650_106.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:28.879650152Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.001689_107.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.001689187Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328769.060092_108.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.060092214Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.162808_109.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.162808526Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328769.343071_110.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.343071979Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328769.600141_111.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.600141569Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328783.361580_112.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.361580865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.503670_113.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.503670361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.576366_114.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.576366221Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.715625_115.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.715625577Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.472869_116.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.472869224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.742386_117.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.742386838Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.812919_118.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.812919362Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.979340_119.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.979340935Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328785.869729_120.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:45.869729749Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.066327_121.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:46.066327601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328786.205110_122.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.205110843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.554014_123.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.554014976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328798.827667_124.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:58.827667322Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.003141_125.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:59.003141030Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328799.697584_126.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.697584863Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.894092_127.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.894092024Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328801.238053_128.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.238053772Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328801.559787_129.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.559787262Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:07:20.336 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:20.337 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:20.338 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:23.711 INFO:teuthology.orchestra.run.smithi019.stdout:[{"id":"1_1590328592.508687_53.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.508687920Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328592.917777_55.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.917777969Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328813.921181_64.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:13.921181724Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.103446_65.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:14.103446504Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.184865_66.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.184865631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.400239_67.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.400239762Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.850168_69.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.850168976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.993137_70.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.993137458Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328815.062089_71.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.062089519Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"write"}}},{"id":"1_1590328815.386281_72.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.386281164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.719132_73.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.719132889Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328816.893052_74.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.893052032Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.948640_75.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:16.948640933Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590328817.073639_76.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:17.073639954Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.402320_96.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.402320164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.697935_97.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.697935376Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.752878_98.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.752878712Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.901862_99.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.901862037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.347957_101.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.347957633Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.486510_102.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.486510226Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.577678_103.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.577678314Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.842804_104.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.842804643Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.251362_105.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.251362523Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.399140_106.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.399140495Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.482971_107.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.482971589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.625971_108.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.625971369Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.665732_154.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.665732383Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"write"}}},{"id":"1_1590329883.848009_155.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.848009328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.936799_156.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:03.936799083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"write"}}},{"id":"1_1590329884.074924_157.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:04.074924748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329884.735428_158.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:04.735428932Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.052055_159.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:05.052055284Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329885.210786_160.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.210786752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.548749_161.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.548749753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.537795_162.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.537795828Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"write"}}},{"id":"1_1590329886.669790_163.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.669790405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.726948_164.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:06.726948984Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"write"}}},{"id":"1_1590329887.030128_165.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:07.030128239Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329887.823525_166.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:07.823525444Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.097199_167.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:08.097199890Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329888.329299_168.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.329299790Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.521426_169.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.521426544Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329889.036851_170.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.036851814Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"write"}}},{"id":"1_1590329889.537905_171.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.537905497Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330412.885284_207.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:52.885284053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.167362_208.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.167362293Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.256206_209.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.256206617Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.402854_210.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.402854007Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.582718_211.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.582718645Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"write"}}},{"id":"1_1590330413.778058_212.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.778058231Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330936.244874_222.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.244874907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.530565_223.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.530565189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330936.597653_224.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.597653809Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.762225_225.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.762225748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.505378_245.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.505378326Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"write"}}},{"id":"1_1590331468.710798_246.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.710798130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.843043_247.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:28.843043489Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590331469.009382_248.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:29.009382468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331469.800887_249.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:29.800887834Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.193693_250.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:30.193693841Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331470.301973_251.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.301973037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.455201_252.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.455201532Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331471.584345_253.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.584345083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"write"}}},{"id":"1_1590331471.850957_254.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.850957150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.003811_255.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.003811342Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"write"}}},{"id":"1_1590331472.200489_256.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.200489331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.703779_257.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:32.703779771Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.055080_258.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:33.055080161Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.401514_259.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.401514029Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.624466_260.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.624466894Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.927577_261.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:33.927577797Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"write"}}},{"id":"1_1590331474.143509_262.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:34.143509836Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332017.477105_282.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.477105632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.591917_283.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.591917171Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332017.652491_284.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.652491494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.767809_285.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.767809971Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332018.626827_286.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.626827980Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332018.983994_287.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.983994356Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332019.048026_288.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.048026105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"write"}}},{"id":"1_1590332019.311787_289.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.311787349Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.450116_290.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.450116581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.603092_291.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.603092627Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.702109_292.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.702109684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.816758_293.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.816758240Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332021.409529_295.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.409529984Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332021.814058_296.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.814058013Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332021.994940_297.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.994940878Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"write"}}},{"id":"1_1590332022.279636_298.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:42.279636992Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332022.525693_299.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.525693290Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"write"}}},{"id":"1_1590332022.817665_300.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.817665067Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332023.133730_301.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.133730721Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"write"}}},{"id":"1_1590332023.446055_302.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.446055762Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332024.079336_303.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.079336921Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332024.395330_304.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.395330949Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332024.698623_306.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.698623067Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"write"}}},{"id":"1_1590332025.003642_307.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:45.003642510Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332576.308624_331.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.308624985Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.530859_332.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.530859626Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332576.609981_333.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.609981019Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.781799_334.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.781799684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.537460_335.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.537460251Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.728195_336.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.728195704Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.788398_337.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.788398528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.960484_338.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.960484471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.162679_339.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.162679805Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.377176_340.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.377176705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.440183_341.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.440183470Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.573394_342.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.573394776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332580.441736_343.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.441736738Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"write"}}},{"id":"1_1590332580.635748_344.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.635748106Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332581.090074_345.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.090074825Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"write"}}},{"id":"1_1590332581.758248_346.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.758248152Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332582.184585_347.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.184585859Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"write"}}},{"id":"1_1590332582.479754_348.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.479754685Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332583.194378_349.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.194378582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.462964_350.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.462964736Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332583.578177_351.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.578177872Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.730991_352.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.730991086Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332584.691763_353.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.691763614Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332584.947121_354.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.947121755Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332585.103707_355.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.103707310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332585.232605_356.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.232605182Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.344013_357.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.344013365Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.464539_358.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.464539687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.582866_359.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.582866987Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.748023_360.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.748023907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332587.677153_361.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.677153663Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"write"}}},{"id":"1_1590332587.932544_362.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.932544635Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332588.327537_363.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.327537080Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"write"}}},{"id":"1_1590332588.619061_364.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.619061527Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332589.216798_365.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.216798093Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"write"}}},{"id":"1_1590332589.561035_366.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.561035178Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332590.255025_367.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.255025912Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.446227_368.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.446227608Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332590.502559_369.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.502559078Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.649027_370.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.649027123Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.375800_371.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.375800124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332591.742382_372.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.742382237Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.842552_373.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:11.842552021Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.031908_374.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:12.031908659Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.240469_375.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.240469031Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.391002_376.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.391002471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.465847_377.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.465847570Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.685001_378.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.685001719Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332594.450933_379.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.450933305Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"write"}}},{"id":"1_1590332594.858645_380.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.858645158Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332596.126321_381.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.126321558Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"write"}}},{"id":"1_1590332596.379140_382.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.379140830Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332597.929328_383.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:17.929328464Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"write"}}},{"id":"1_1590332598.267606_384.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:18.267606593Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332598.971437_385.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:18.971437370Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.187031_386.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:19.187031446Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332599.250412_387.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.250412420Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.365739_388.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.365739368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.095396_389.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.095396102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.271093_390.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.271093243Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.332512_391.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.332512615Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.459857_392.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.459857112Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332601.952044_393.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:21.952044861Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.098949_394.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:22.098949881Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332602.200693_395.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.200693657Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.324881_396.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.324881490Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332618.489788_398.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:38.489788797Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332619.055324_399.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:39.055324511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332619.292390_400.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:39.292390413Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332620.534454_401.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:40.534454583Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332622.462827_402.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:42.462827592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.169124_403.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:43.169124859Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332623.408767_404.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.408767528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.748848_405.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.748848044Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332624.895711_406.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:44.895711105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.057684_407.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:45.057684752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332625.146752_408.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.146752921Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.395323_409.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.395323934Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328687.177911_34.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:07.177911653Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328692.069642_35.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:12.069642200Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328692.266320_36.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:12.266320581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328693.128004_37.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:13.128004764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.151011_38.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:15.151011218Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.949640_39.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:17.949640848Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328700.795017_40.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:20.795017810Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328702.912912_41.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:22.912912206Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.578055_42.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.578055891Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.729955_43.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.729955016Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.799034_44.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.799034852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.981332_45.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.981332310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.253514_46.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.253514189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.389746_47.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.389746469Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.462701_48.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.462701880Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.646948_49.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.646948705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.108322_50.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.108322825Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.278473_51.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.278473753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.347900_52.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.347900543Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.546580_53.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.546580127Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328742.949663_54.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:02.949663616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.100429_55.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:03.100429758Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328743.339199_56.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.339199274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.504813_57.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.504813479Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.417840_58.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.417840348Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.529097_59.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.529097974Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.586991_60.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.586991054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.713299_61.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.713299225Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328756.828867_62.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.828867828Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328757.071284_63.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:17.071284402Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328757.495644_64.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.495644501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.636125_65.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.636125563Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328757.719832_66.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.719832184Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.845993_67.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.845993517Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328758.093585_68.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.093585822Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328758.317456_69.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.317456261Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328758.691521_70.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.691521521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328758.854008_71.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.854008793Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328758.982217_72.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:18.982217287Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"write"}}},{"id":"1_1590328759.162486_73.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:19.162486093Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328759.283606_74.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.283606303Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328759.467567_75.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.467567593Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328760.133426_76.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.133426441Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.326796_77.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.326796578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328760.396609_78.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.396609688Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.501441_79.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.501441090Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328760.605321_80.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.605321185Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328760.874531_81.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.874531368Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328761.934150_82.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:21.934150091Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.135705_83.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.135705114Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328762.220642_84.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.220642897Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.351162_85.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.351162800Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328762.451877_86.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.451877182Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328762.616596_87.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.616596068Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328763.544481_88.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.544481776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"write"}}},{"id":"1_1590328763.780813_89.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.780813494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328763.875460_90.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:23.875460851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"write"}}},{"id":"1_1590328764.115279_91.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:24.115279014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328764.302206_92.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.302206541Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.586734_93.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.586734819Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.603699_94.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.603699796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328765.804758_95.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.804758399Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328765.871314_96.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:25.871314468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"write"}}},{"id":"1_1590328766.001613_97.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:26.001613252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328766.178702_98.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.178702867Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328766.369723_99.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.369723003Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.147013_100.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.147013040Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.356936_101.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.356936965Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328767.419119_102.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.419119272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.609379_103.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.609379045Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328767.717880_104.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.717880724Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.877492_105.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.877492811Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328768.879650_106.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:28.879650152Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.001689_107.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.001689187Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328769.060092_108.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.060092214Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.162808_109.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.162808526Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328769.343071_110.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.343071979Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328769.600141_111.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.600141569Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328783.361580_112.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.361580865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.503670_113.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.503670361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.576366_114.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.576366221Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.715625_115.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.715625577Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.472869_116.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.472869224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.742386_117.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.742386838Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.812919_118.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.812919362Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.979340_119.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.979340935Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328785.869729_120.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:45.869729749Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.066327_121.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:46.066327601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328786.205110_122.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.205110843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.554014_123.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.554014976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328798.827667_124.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:58.827667322Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.003141_125.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:59.003141030Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328799.697584_126.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.697584863Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.894092_127.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.894092024Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328801.238053_128.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.238053772Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328801.559787_129.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.559787262Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:07:23.715 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:23.715 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:23.716 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:26.730 INFO:teuthology.orchestra.run.smithi193.stdout:[{"id":"1_1590328592.508687_53.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.508687920Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328592.917777_55.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:32.917777969Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xnx4DcU4PDV302bW3bsz8DZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328813.921181_64.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:13.921181724Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.103446_65.1","section":"bucket.instance","name":"fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","timestamp":"2020-05-24 14:00:14.103446504Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DB9alC2HsL2RGKrFE80gCGf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.184865_66.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.184865631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.400239_67.1","section":"bucket","name":"fwzbiz-10","timestamp":"2020-05-24 14:00:14.400239762Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7tBby65ZRdQA62AiqSWLEv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328814.850168_69.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.850168976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328814.993137_70.1","section":"bucket.instance","name":"fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","timestamp":"2020-05-24 14:00:14.993137458Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lKE0Mn6daIiY6EDLx8pmxR6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328815.062089_71.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.062089519Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"write"}}},{"id":"1_1590328815.386281_72.1","section":"bucket","name":"fwzbiz-11","timestamp":"2020-05-24 14:00:15.386281164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5OSaYAeqS_UB0uD3NL96dlS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.719132_73.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.719132889Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328816.893052_74.1","section":"bucket.instance","name":"fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","timestamp":"2020-05-24 14:00:16.893052032Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_A7ac20e34V-khCYHFd7BTVk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328816.948640_75.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:16.948640933Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590328817.073639_76.1","section":"bucket","name":"fwzbiz-12","timestamp":"2020-05-24 14:00:17.073639954Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CONEvg-_ZEwvKUmn1Zp_Y5Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.402320_96.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.402320164Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.697935_97.1","section":"bucket.instance","name":"fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","timestamp":"2020-05-24 14:09:18.697935376Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QaFRJTYVM9Ebn8nO3klJfBF","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329358.752878_98.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.752878712Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"write"}}},{"id":"1_1590329358.901862_99.1","section":"bucket","name":"fwzbiz-13","timestamp":"2020-05-24 14:09:18.901862037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Z41kVO1t6H567XZGXmyi5Ac","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.347957_101.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.347957633Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.486510_102.1","section":"bucket.instance","name":"fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","timestamp":"2020-05-24 14:09:19.486510226Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2N5TAgVEZZ_JOjsT2awRQUm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329359.577678_103.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.577678314Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329359.842804_104.1","section":"bucket","name":"fwzbiz-14","timestamp":"2020-05-24 14:09:19.842804643Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5MB_uBVtp4kWawLkF6EYh9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.251362_105.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.251362523Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.399140_106.1","section":"bucket.instance","name":"fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","timestamp":"2020-05-24 14:09:21.399140495Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SDxV_47sxtZRUQsQ5V5hJP7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329361.482971_107.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.482971589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"write"}}},{"id":"1_1590329361.625971_108.1","section":"bucket","name":"fwzbiz-15","timestamp":"2020-05-24 14:09:21.625971369Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXcGAqEAOK_4Ym80o4Ss1nf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.665732_154.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.665732383Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"write"}}},{"id":"1_1590329883.848009_155.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:03.848009328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329883.936799_156.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:03.936799083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"write"}}},{"id":"1_1590329884.074924_157.1","section":"bucket","name":"fwzbiz-16","timestamp":"2020-05-24 14:18:04.074924748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BrkUsBO1tQCgTy6-VYvyeCB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329884.735428_158.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:04.735428932Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.052055_159.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:05.052055284Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329885.210786_160.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.210786752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"write"}}},{"id":"1_1590329885.548749_161.1","section":"bucket","name":"fwzbiz-17","timestamp":"2020-05-24 14:18:05.548749753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7ZiaBJkttG_klSGBLD5v9kr","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.537795_162.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.537795828Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"write"}}},{"id":"1_1590329886.669790_163.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:06.669790405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329886.726948_164.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:06.726948984Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"write"}}},{"id":"1_1590329887.030128_165.1","section":"bucket","name":"fwzbiz-18","timestamp":"2020-05-24 14:18:07.030128239Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qOzEtlGs1lJLh_dI49Cwe02","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329887.823525_166.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:07.823525444Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.097199_167.1","section":"bucket.instance","name":"fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","timestamp":"2020-05-24 14:18:08.097199890Z","data":{"read_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":1},"write_version":{"tag":"_rls_y25CkDDiyrpC9aAJXi8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329888.329299_168.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.329299790Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"write"}}},{"id":"1_1590329888.521426_169.1","section":"bucket.instance","name":"fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","timestamp":"2020-05-24 14:18:08.521426544Z","data":{"read_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":1},"write_version":{"tag":"_WDcn1qOoPhVuvJ2OiVFmtmZ","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329889.036851_170.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.036851814Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"write"}}},{"id":"1_1590329889.537905_171.1","section":"bucket.instance","name":"fwzbiz-18:4e214d8a-46aa-488b-8afd-3016515a7228.4387.27","timestamp":"2020-05-24 14:18:09.537905497Z","data":{"read_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":1},"write_version":{"tag":"_9MA84Dq-ObvgahjhknQasJV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330412.885284_207.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:52.885284053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.167362_208.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.167362293Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.256206_209.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.256206617Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"write"}}},{"id":"1_1590330413.402854_210.1","section":"bucket","name":"fwzbiz-19","timestamp":"2020-05-24 14:26:53.402854007Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dlzSGYjinAB5b2ItCxPAY5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330413.582718_211.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.582718645Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"write"}}},{"id":"1_1590330413.778058_212.1","section":"bucket.instance","name":"fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","timestamp":"2020-05-24 14:26:53.778058231Z","data":{"read_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":1},"write_version":{"tag":"_ZZQIaOvzD1A5BE2yZDuPm-V","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330936.244874_222.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.244874907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.530565_223.1","section":"bucket.instance","name":"fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","timestamp":"2020-05-24 14:35:36.530565189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_r3BdmRgaGzdFTxSlz85uYvC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330936.597653_224.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.597653809Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"write"}}},{"id":"1_1590330936.762225_225.1","section":"bucket","name":"fwzbiz-20","timestamp":"2020-05-24 14:35:36.762225748Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_jCGCXBSrakZfODbSKRJXk7E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.505378_245.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.505378326Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"write"}}},{"id":"1_1590331468.710798_246.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:28.710798130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331468.843043_247.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:28.843043489Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590331469.009382_248.1","section":"bucket","name":"fwzbiz-21","timestamp":"2020-05-24 14:44:29.009382468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gPjDrLYmNLh5eEmtK_lIe9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331469.800887_249.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:29.800887834Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.193693_250.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:30.193693841Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331470.301973_251.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.301973037Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"write"}}},{"id":"1_1590331470.455201_252.1","section":"bucket","name":"fwzbiz-22","timestamp":"2020-05-24 14:44:30.455201532Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IMQXMo7JwSz7OL9G6V9Y36S","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331471.584345_253.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.584345083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"write"}}},{"id":"1_1590331471.850957_254.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:31.850957150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.003811_255.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.003811342Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"write"}}},{"id":"1_1590331472.200489_256.1","section":"bucket","name":"fwzbiz-23","timestamp":"2020-05-24 14:44:32.200489331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mMw9Fwg6k4MJEURKqu8BWmi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331472.703779_257.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:32.703779771Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.055080_258.1","section":"bucket.instance","name":"fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","timestamp":"2020-05-24 14:44:33.055080161Z","data":{"read_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":1},"write_version":{"tag":"_RdQSmhfS4XeKhSw8AWdlV68","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.401514_259.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.401514029Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"write"}}},{"id":"1_1590331473.624466_260.1","section":"bucket.instance","name":"fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","timestamp":"2020-05-24 14:44:33.624466894Z","data":{"read_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":1},"write_version":{"tag":"_ku3VYgj83m87x8X2Blhn_Ie","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331473.927577_261.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:33.927577797Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"write"}}},{"id":"1_1590331474.143509_262.1","section":"bucket.instance","name":"fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","timestamp":"2020-05-24 14:44:34.143509836Z","data":{"read_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":1},"write_version":{"tag":"_HzuOeMOqo8sYxq_saxVbLtj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332017.477105_282.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.477105632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.591917_283.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:37.591917171Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332017.652491_284.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.652491494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332017.767809_285.1","section":"bucket","name":"fwzbiz-24","timestamp":"2020-05-24 14:53:37.767809971Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_KI7rA6oibUlgMEmY3uUpTmg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332018.626827_286.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.626827980Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332018.983994_287.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:38.983994356Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332019.048026_288.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.048026105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"write"}}},{"id":"1_1590332019.311787_289.1","section":"bucket","name":"fwzbiz-25","timestamp":"2020-05-24 14:53:39.311787349Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RJQuISPxgYef8XR2cyghO3Y","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.450116_290.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.450116581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.603092_291.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:40.603092627Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332020.702109_292.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.702109684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332020.816758_293.1","section":"bucket","name":"fwzbiz-26","timestamp":"2020-05-24 14:53:40.816758240Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7SscGBNWvgAwySQGUodl5DX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332021.409529_295.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.409529984Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332021.814058_296.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.814058013Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":1},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332021.994940_297.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:41.994940878Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"write"}}},{"id":"1_1590332022.279636_298.1","section":"bucket.instance","name":"fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","timestamp":"2020-05-24 14:53:42.279636992Z","data":{"read_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":2},"write_version":{"tag":"_rXpAeGSmwNfa_gKAZDRR9LM","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332022.525693_299.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.525693290Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"write"}}},{"id":"1_1590332022.817665_300.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:42.817665067Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":1},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332023.133730_301.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.133730721Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"write"}}},{"id":"1_1590332023.446055_302.1","section":"bucket.instance","name":"fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","timestamp":"2020-05-24 14:53:43.446055762Z","data":{"read_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":2},"write_version":{"tag":"_lDZAT_hQzTdZlzo8vDSDwYV","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332024.079336_303.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.079336921Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332024.395330_304.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.395330949Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":1},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332024.698623_306.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:44.698623067Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"write"}}},{"id":"1_1590332025.003642_307.1","section":"bucket.instance","name":"fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","timestamp":"2020-05-24 14:53:45.003642510Z","data":{"read_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":2},"write_version":{"tag":"_XXEv0FVrDzxISN9NfQ32fsG","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332576.308624_331.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.308624985Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.530859_332.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:02:56.530859626Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332576.609981_333.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.609981019Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"write"}}},{"id":"1_1590332576.781799_334.1","section":"bucket","name":"fwzbiz-27","timestamp":"2020-05-24 15:02:56.781799684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OANM2sASB_0reoxsVzskS9W","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.537460_335.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.537460251Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.728195_336.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:02:57.728195704Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332577.788398_337.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.788398528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"write"}}},{"id":"1_1590332577.960484_338.1","section":"bucket","name":"fwzbiz-28","timestamp":"2020-05-24 15:02:57.960484471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_goFei88J0I9IDgske-EqkdN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.162679_339.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.162679805Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.377176_340.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:02:59.377176705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332579.440183_341.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.440183470Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332579.573394_342.1","section":"bucket","name":"fwzbiz-29","timestamp":"2020-05-24 15:02:59.573394776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2rPnJ0o_TYrLb-HpeJC_5Pb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332580.441736_343.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.441736738Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"write"}}},{"id":"1_1590332580.635748_344.1","section":"bucket.instance","name":"fwzbiz-27:4e214d8a-46aa-488b-8afd-3016515a7228.4387.36","timestamp":"2020-05-24 15:03:00.635748106Z","data":{"read_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":1},"write_version":{"tag":"_Ew69vVShXcw1YuNT92tOu5I","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332581.090074_345.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.090074825Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"write"}}},{"id":"1_1590332581.758248_346.1","section":"bucket.instance","name":"fwzbiz-28:4e214d8a-46aa-488b-8afd-3016515a7228.4387.37","timestamp":"2020-05-24 15:03:01.758248152Z","data":{"read_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":1},"write_version":{"tag":"_zLXFy9ZKXS5smh6YGxRjab_","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332582.184585_347.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.184585859Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"write"}}},{"id":"1_1590332582.479754_348.1","section":"bucket.instance","name":"fwzbiz-29:4e214d8a-46aa-488b-8afd-3016515a7228.4387.38","timestamp":"2020-05-24 15:03:02.479754685Z","data":{"read_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":1},"write_version":{"tag":"_ROJV9q0ZXweOA4f6vjqZLpc","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332583.194378_349.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.194378582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.462964_350.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:03.462964736Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332583.578177_351.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.578177872Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"write"}}},{"id":"1_1590332583.730991_352.1","section":"bucket","name":"fwzbiz-30","timestamp":"2020-05-24 15:03:03.730991086Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YmpkWvqk2DtybHuQJmqZOyT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332584.691763_353.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.691763614Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332584.947121_354.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:04.947121755Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332585.103707_355.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.103707310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"write"}}},{"id":"1_1590332585.232605_356.1","section":"bucket","name":"fwzbiz-31","timestamp":"2020-05-24 15:03:05.232605182Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LfnxwtHCdb5Cq8ML4iAwuWb","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.344013_357.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.344013365Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.464539_358.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:06.464539687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332586.582866_359.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.582866987Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332586.748023_360.1","section":"bucket","name":"fwzbiz-32","timestamp":"2020-05-24 15:03:06.748023907Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BPFU3aLhHdsLcifRd4pMFU3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332587.677153_361.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.677153663Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"write"}}},{"id":"1_1590332587.932544_362.1","section":"bucket.instance","name":"fwzbiz-30:4e214d8a-46aa-488b-8afd-3016515a7228.4387.39","timestamp":"2020-05-24 15:03:07.932544635Z","data":{"read_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":1},"write_version":{"tag":"_FKZ-mME-cK5J1Wjc2cn9TIO","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332588.327537_363.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.327537080Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"write"}}},{"id":"1_1590332588.619061_364.1","section":"bucket.instance","name":"fwzbiz-31:4e214d8a-46aa-488b-8afd-3016515a7228.4387.40","timestamp":"2020-05-24 15:03:08.619061527Z","data":{"read_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":1},"write_version":{"tag":"_mhkFWNPKIsDsfPb_nxsiFaK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332589.216798_365.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.216798093Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"write"}}},{"id":"1_1590332589.561035_366.1","section":"bucket.instance","name":"fwzbiz-32:4e214d8a-46aa-488b-8afd-3016515a7228.4387.41","timestamp":"2020-05-24 15:03:09.561035178Z","data":{"read_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":1},"write_version":{"tag":"_HtfkP0yqBDaBmSnx6XPeIoY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332590.255025_367.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.255025912Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.446227_368.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:10.446227608Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332590.502559_369.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.502559078Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332590.649027_370.1","section":"bucket","name":"fwzbiz-33","timestamp":"2020-05-24 15:03:10.649027123Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C-uHsUw3hPPc2MrPk6Ok6xi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.375800_371.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.375800124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"write"}}},{"id":"1_1590332591.742382_372.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:11.742382237Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.842552_373.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:11.842552021Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.031908_374.1","section":"bucket","name":"fwzbiz-34","timestamp":"2020-05-24 15:03:12.031908659Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zQh3EhOyRXJGw-cYuETSmhJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.240469_375.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.240469031Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.391002_376.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:13.391002471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.465847_377.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.465847570Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.685001_378.1","section":"bucket","name":"fwzbiz-35","timestamp":"2020-05-24 15:03:13.685001719Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W_J1uK2ipKTHxOkgqUZPgtG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332594.450933_379.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.450933305Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"write"}}},{"id":"1_1590332594.858645_380.1","section":"bucket.instance","name":"fwzbiz-33:4e214d8a-46aa-488b-8afd-3016515a7228.4387.42","timestamp":"2020-05-24 15:03:14.858645158Z","data":{"read_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":1},"write_version":{"tag":"_7sgvIueaddaj3FwC0khtBOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332596.126321_381.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.126321558Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"write"}}},{"id":"1_1590332596.379140_382.1","section":"bucket.instance","name":"fwzbiz-34:4e214d8a-46aa-488b-8afd-3016515a7228.4387.43","timestamp":"2020-05-24 15:03:16.379140830Z","data":{"read_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":1},"write_version":{"tag":"_-dR-29RWZZpx6NWRn3A7nHX","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332597.929328_383.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:17.929328464Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"write"}}},{"id":"1_1590332598.267606_384.1","section":"bucket.instance","name":"fwzbiz-35:4e214d8a-46aa-488b-8afd-3016515a7228.4387.44","timestamp":"2020-05-24 15:03:18.267606593Z","data":{"read_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":1},"write_version":{"tag":"_SV-oD4LT4gdBUqXixhTeB2l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332598.971437_385.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:18.971437370Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.187031_386.1","section":"bucket.instance","name":"fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","timestamp":"2020-05-24 15:03:19.187031446Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7sth6XwvbU56QOv0pudQPS8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332599.250412_387.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.250412420Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"write"}}},{"id":"1_1590332599.365739_388.1","section":"bucket","name":"fwzbiz-36","timestamp":"2020-05-24 15:03:19.365739368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_15gR7Gq0RxmG39eMozVaP9E","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.095396_389.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.095396102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.271093_390.1","section":"bucket.instance","name":"fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","timestamp":"2020-05-24 15:03:20.271093243Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4Grxk9fadY6iovSOcqRw2AL","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332600.332512_391.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.332512615Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"write"}}},{"id":"1_1590332600.459857_392.1","section":"bucket","name":"fwzbiz-37","timestamp":"2020-05-24 15:03:20.459857112Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7tRhgTnOSJqK4Oodj0wYLNd","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332601.952044_393.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:21.952044861Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.098949_394.1","section":"bucket.instance","name":"fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","timestamp":"2020-05-24 15:03:22.098949881Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y4HN1tJuWZSWGh8zcHf_0-8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332602.200693_395.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.200693657Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590332602.324881_396.1","section":"bucket","name":"fwzbiz-38","timestamp":"2020-05-24 15:03:22.324881490Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YaS2ikaXm1ZW9Y0S7N6g1mZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332618.489788_398.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:38.489788797Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332619.055324_399.1","section":"bucket.instance","name":"fwzbiz-39:4e214d8a-46aa-488b-8afd-3016515a7228.4387.48","timestamp":"2020-05-24 15:03:39.055324511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OnrG_adtkqlcDUIdfOj0Xi1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332619.292390_400.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:39.292390413Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"write"}}},{"id":"1_1590332620.534454_401.1","section":"bucket","name":"fwzbiz-39","timestamp":"2020-05-24 15:03:40.534454583Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_svkd7H_5EqxAM1m3A2LktEi","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332622.462827_402.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:42.462827592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.169124_403.1","section":"bucket.instance","name":"fwzbiz-40:4e214d8a-46aa-488b-8afd-3016515a7228.4387.49","timestamp":"2020-05-24 15:03:43.169124859Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3q9nDfycgw1bG4ovq4XjoYH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332623.408767_404.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.408767528Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"write"}}},{"id":"1_1590332623.748848_405.1","section":"bucket","name":"fwzbiz-40","timestamp":"2020-05-24 15:03:43.748848044Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_l-og4knKJdbzNiP0f8kosYB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332624.895711_406.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:44.895711105Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.057684_407.1","section":"bucket.instance","name":"fwzbiz-41:4e214d8a-46aa-488b-8afd-3016515a7228.4387.50","timestamp":"2020-05-24 15:03:45.057684752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RQhYwasQz6BE2DS_YRd3rd1","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332625.146752_408.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.146752921Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"write"}}},{"id":"1_1590332625.395323_409.1","section":"bucket","name":"fwzbiz-41","timestamp":"2020-05-24 15:03:45.395323934Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WEuHsNahrDqGt5tV9CzTSDD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328687.177911_34.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:07.177911653Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"write"}}},{"id":"1_1590328692.069642_35.1","section":"bucket.instance","name":"fwzbiz-1:4e214d8a-46aa-488b-8afd-3016515a7228.4387.1","timestamp":"2020-05-24 13:58:12.069642200Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_edgdu88RX026apwkRuu8sLv","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328692.266320_36.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:12.266320581Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328693.128004_37.1","section":"bucket","name":"fwzbiz-1","timestamp":"2020-05-24 13:58:13.128004764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LE3NBQQ6axNZ9n1IOE1Qxuq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.151011_38.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:15.151011218Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.949640_39.1","section":"bucket.instance","name":"fwzbiz-2:4e214d8a-46aa-488b-8afd-3016515a7228.4387.2","timestamp":"2020-05-24 13:58:17.949640848Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9K3dy1ofS6cBNrCYX_8ON9q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328700.795017_40.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:20.795017810Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590328702.912912_41.1","section":"bucket","name":"fwzbiz-2","timestamp":"2020-05-24 13:58:22.912912206Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SXxQlDxXQl_G-a_7mulPZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.578055_42.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.578055891Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.729955_43.1","section":"bucket.instance","name":"fwzbiz-3:4e214d8a-46aa-488b-8afd-3016515a7228.4387.3","timestamp":"2020-05-24 13:58:32.729955016Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VT8y1fZzGKBxbWgGrsrH89Q","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328712.799034_44.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.799034852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"write"}}},{"id":"1_1590328712.981332_45.1","section":"bucket","name":"fwzbiz-3","timestamp":"2020-05-24 13:58:32.981332310Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_8dMfuJGoLMOhKLIaxPAc779","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.253514_46.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.253514189Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.389746_47.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.4","timestamp":"2020-05-24 13:59:00.389746469Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GG1b03YqrDo3CQTI_Ay3Fli","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328740.462701_48.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.462701880Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"write"}}},{"id":"1_1590328740.646948_49.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:00.646948705Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bxVAjcYsZKTkSIiRwNX0HKN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.108322_50.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.108322825Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.278473_51.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.5","timestamp":"2020-05-24 13:59:01.278473753Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JOkvtesLQ2bSaw2h25h7Shm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328741.347900_52.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.347900543Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"write"}}},{"id":"1_1590328741.546580_53.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:01.546580127Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vcy6iLADBVTEZVGjA7MyMAC","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328742.949663_54.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:02.949663616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.100429_55.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.6","timestamp":"2020-05-24 13:59:03.100429758Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TLw8sUzi4su4FmeRyB4kFkU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328743.339199_56.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.339199274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"write"}}},{"id":"1_1590328743.504813_57.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:03.504813479Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_oGnkmDN7_7R95rB02gF9XOB","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.417840_58.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.417840348Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.529097_59.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.529097974Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328756.586991_60.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.586991054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"write"}}},{"id":"1_1590328756.713299_61.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:16.713299225Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qa-iQPpVjbY8wz6XJLK_QBT","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328756.828867_62.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:16.828867828Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328757.071284_63.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.7","timestamp":"2020-05-24 13:59:17.071284402Z","data":{"read_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":1},"write_version":{"tag":"_M8zNVo0fVKy8JiJrymwBu15","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328757.495644_64.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.495644501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.636125_65.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:17.636125563Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328757.719832_66.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.719832184Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"write"}}},{"id":"1_1590328757.845993_67.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:17.845993517Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-d1SSuhP9ucQtIdHtqZeP9H","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328758.093585_68.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.093585822Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328758.317456_69.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.8","timestamp":"2020-05-24 13:59:18.317456261Z","data":{"read_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":1},"write_version":{"tag":"_MnPh15Yi_FK9xu31zRO6JyH","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328758.691521_70.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.691521521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328758.854008_71.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:18.854008793Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328758.982217_72.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:18.982217287Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"write"}}},{"id":"1_1590328759.162486_73.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:19.162486093Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_srhEAnVvV3h1vZ3O2ScuPyg","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328759.283606_74.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.283606303Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328759.467567_75.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.9","timestamp":"2020-05-24 13:59:19.467567593Z","data":{"read_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":1},"write_version":{"tag":"_YNPSpgd0GhbhEgcvWKNJWtM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328760.133426_76.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.133426441Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.326796_77.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.326796578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328760.396609_78.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.396609688Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328760.501441_79.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:20.501441090Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_SW1cDkVxM4OJHkHakjcZIou","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328760.605321_80.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.605321185Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328760.874531_81.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.10","timestamp":"2020-05-24 13:59:20.874531368Z","data":{"read_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":1},"write_version":{"tag":"_FrLyh-E_b7KMTKPDqHOYnt8","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328761.934150_82.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:21.934150091Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.135705_83.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.135705114Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328762.220642_84.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.220642897Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328762.351162_85.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:22.351162800Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hEzYE4ItEQARiB-SHFI9pzK","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328762.451877_86.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.451877182Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328762.616596_87.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.11","timestamp":"2020-05-24 13:59:22.616596068Z","data":{"read_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":1},"write_version":{"tag":"_I3PW4b1vX2tszPRmM0V0eeU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328763.544481_88.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.544481776Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"write"}}},{"id":"1_1590328763.780813_89.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:23.780813494Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328763.875460_90.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:23.875460851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"write"}}},{"id":"1_1590328764.115279_91.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:24.115279014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-NKF8nz7trGBdWs768JIG3s","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328764.302206_92.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.302206541Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.586734_93.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.12","timestamp":"2020-05-24 13:59:24.586734819Z","data":{"read_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":1},"write_version":{"tag":"_aYx5NkmGv6ZUjJdNoZJUqel","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.603699_94.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.603699796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"write"}}},{"id":"1_1590328765.804758_95.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:25.804758399Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328765.871314_96.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:25.871314468Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"write"}}},{"id":"1_1590328766.001613_97.1","section":"bucket","name":"fwzbiz-4","timestamp":"2020-05-24 13:59:26.001613252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_hl49nVxXVzIB26nt2O5cUDP","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328766.178702_98.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.178702867Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328766.369723_99.1","section":"bucket.instance","name":"fwzbiz-4:4e214d8a-46aa-488b-8afd-3016515a7228.4387.13","timestamp":"2020-05-24 13:59:26.369723003Z","data":{"read_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":1},"write_version":{"tag":"_XaV5xOr9x9EuuO3RG3bvqUq","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.147013_100.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.147013040Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.356936_101.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.356936965Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328767.419119_102.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.419119272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"write"}}},{"id":"1_1590328767.609379_103.1","section":"bucket","name":"fwzbiz-5","timestamp":"2020-05-24 13:59:27.609379045Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ijZzhov9K69tZeVk4a1RdLA","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328767.717880_104.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.717880724Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.877492_105.1","section":"bucket.instance","name":"fwzbiz-5:4e214d8a-46aa-488b-8afd-3016515a7228.4387.14","timestamp":"2020-05-24 13:59:27.877492811Z","data":{"read_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":1},"write_version":{"tag":"_DWE72b__YTWWq0AfvPOCevK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328768.879650_106.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:28.879650152Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.001689_107.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.001689187Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328769.060092_108.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.060092214Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"write"}}},{"id":"1_1590328769.162808_109.1","section":"bucket","name":"fwzbiz-6","timestamp":"2020-05-24 13:59:29.162808526Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_T53Ovy6I5_r8i6kj0pcXNZh","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328769.343071_110.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.343071979Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328769.600141_111.1","section":"bucket.instance","name":"fwzbiz-6:4e214d8a-46aa-488b-8afd-3016515a7228.4387.15","timestamp":"2020-05-24 13:59:29.600141569Z","data":{"read_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":1},"write_version":{"tag":"_I2ba9iAmViveQvyzW8creYa","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328783.361580_112.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.361580865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.503670_113.1","section":"bucket.instance","name":"fwzbiz-7:4e214d8a-46aa-488b-8afd-3016515a7228.4387.16","timestamp":"2020-05-24 13:59:43.503670361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4oa4ISxr_a6uBVQ2ESRv_Xo","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.576366_114.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.576366221Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.715625_115.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:43.715625577Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.472869_116.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.472869224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.742386_117.1","section":"bucket.instance","name":"fwzbiz-8:4e214d8a-46aa-488b-8afd-3016515a7228.4387.17","timestamp":"2020-05-24 13:59:44.742386838Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PNNdMcaHikhVnrQfIHXiLou","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328784.812919_118.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.812919362Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328784.979340_119.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:44.979340935Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328785.869729_120.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:45.869729749Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.066327_121.1","section":"bucket.instance","name":"fwzbiz-9:4e214d8a-46aa-488b-8afd-3016515a7228.4387.18","timestamp":"2020-05-24 13:59:46.066327601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_v8DrueZ22kF930bAIpGzn6f","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328786.205110_122.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.205110843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"write"}}},{"id":"1_1590328786.554014_123.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 13:59:46.554014976Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328798.827667_124.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:58.827667322Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.003141_125.1","section":"bucket","name":"fwzbiz-7","timestamp":"2020-05-24 13:59:59.003141030Z","data":{"read_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":1},"write_version":{"tag":"_MbpHFAZw9W1kxY6E_KMjU9l","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328799.697584_126.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.697584863Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328799.894092_127.1","section":"bucket","name":"fwzbiz-8","timestamp":"2020-05-24 13:59:59.894092024Z","data":{"read_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":1},"write_version":{"tag":"_W8H0mYXODc5cGys6ia13ZaW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328801.238053_128.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.238053772Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328801.559787_129.1","section":"bucket","name":"fwzbiz-9","timestamp":"2020-05-24 14:00:01.559787262Z","data":{"read_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":1},"write_version":{"tag":"_xvps9bdueMOKkVmBxh-pB59","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:07:26.732 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:26.733 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:26.734 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:29.899 INFO:teuthology.orchestra.run.smithi019.stdout:[{"id":"1_1590332709.148482_134.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.148482823Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.285780_135.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.285780687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332709.373824_136.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.373824151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.489583_137.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.489583294Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.550132_138.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.550132862Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.702462_139.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.702462281Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.783850_140.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.783850637Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.970452_141.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.970452384Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:29.900 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:29.901 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:29.901 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:32.471 INFO:teuthology.orchestra.run.smithi193.stdout:[{"id":"1_1590332709.148482_134.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.148482823Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.285780_135.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.285780687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332709.373824_136.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.373824151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.489583_137.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.489583294Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.550132_138.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.550132862Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.702462_139.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.702462281Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.783850_140.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.783850637Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.970452_141.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.970452384Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:32.472 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:32.473 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:32.474 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:35.531 INFO:teuthology.orchestra.run.smithi019.stdout:[{"id":"1_1590332709.148482_134.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.148482823Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.285780_135.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.285780687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332709.373824_136.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.373824151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.489583_137.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.489583294Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.550132_138.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.550132862Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.702462_139.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.702462281Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.783850_140.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.783850637Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.970452_141.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.970452384Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:35.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:35.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:35.533 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:38.100 INFO:teuthology.orchestra.run.smithi193.stdout:[{"id":"1_1590332709.148482_134.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.148482823Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.285780_135.1","section":"bucket.instance","name":"fwzbiz-42:692d187d-2dfb-4f37-a732-96d22188c425.8106.1","timestamp":"2020-05-24 15:05:09.285780687Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BZigURrauUYaGM8-FTomCO3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332709.373824_136.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.373824151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332709.489583_137.1","section":"bucket","name":"fwzbiz-42","timestamp":"2020-05-24 15:05:09.489583294Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4FSmuD244Z_opCZzy0i6eSu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.550132_138.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.550132862Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.702462_139.1","section":"bucket.instance","name":"fwzbiz-43:692d187d-2dfb-4f37-a732-96d22188c425.8106.2","timestamp":"2020-05-24 15:05:10.702462281Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-bwyeffg6R8CyuNiP0r_YL2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332710.783850_140.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.783850637Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"write"}}},{"id":"1_1590332710.970452_141.1","section":"bucket","name":"fwzbiz-43","timestamp":"2020-05-24 15:05:10.970452384Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3nRthKx06agaUtnvl0qoNQH","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:38.101 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:38.101 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:38.102 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:41.577 INFO:teuthology.orchestra.run.smithi019.stdout:[{"id":"1_1590332782.379231_89.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.379231556Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.547803_90.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.547803696Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332782.602908_91.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.602908629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.731768_92.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.731768307Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.472793_93.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.472793787Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.628457_94.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.628457972Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.698482_95.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.698482224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.852667_96.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.852667084Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:41.578 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:41.579 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:41.580 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:44.067 INFO:teuthology.orchestra.run.smithi193.stdout:[{"id":"1_1590332782.379231_89.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.379231556Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.547803_90.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.547803696Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332782.602908_91.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.602908629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.731768_92.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.731768307Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.472793_93.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.472793787Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.628457_94.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.628457972Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.698482_95.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.698482224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.852667_96.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.852667084Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:44.067 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:44.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:44.069 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:47.226 INFO:teuthology.orchestra.run.smithi019.stdout:[{"id":"1_1590332782.379231_89.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.379231556Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.547803_90.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.547803696Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332782.602908_91.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.602908629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.731768_92.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.731768307Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.472793_93.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.472793787Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.628457_94.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.628457972Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.698482_95.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.698482224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.852667_96.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.852667084Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:47.227 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:47.227 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:47.228 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:50.168 INFO:teuthology.orchestra.run.smithi193.stdout:[{"id":"1_1590332782.379231_89.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.379231556Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.547803_90.1","section":"bucket.instance","name":"fwzbiz-44:4e214d8a-46aa-488b-8afd-3016515a7228.5043.1","timestamp":"2020-05-24 15:06:22.547803696Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rQyPLvGwis68vJ7lqZZfyKm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332782.602908_91.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.602908629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332782.731768_92.1","section":"bucket","name":"fwzbiz-44","timestamp":"2020-05-24 15:06:22.731768307Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_repRtv7WjIjaLe-D-zzWpX7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.472793_93.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.472793787Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.628457_94.1","section":"bucket.instance","name":"fwzbiz-45:4e214d8a-46aa-488b-8afd-3016515a7228.5043.2","timestamp":"2020-05-24 15:06:23.628457972Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_qofS_bHGcAl3eJa1NNcLGsg","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332783.698482_95.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.698482224Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"write"}}},{"id":"1_1590332783.852667_96.1","section":"bucket","name":"fwzbiz-45","timestamp":"2020-05-24 15:06:23.852667084Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JiFEHEmGNqg4zXgR4Gm3lgw","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:50.169 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:50.169 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:50.170 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:07:54.632 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:54.633 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:54.634 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:07:59.339 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:59.340 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:59.341 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:08:01.609 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:19:11.829 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:08:01.612 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:19:11.832 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:08:01.822 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:19:12.031 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:08:03.322 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:08:03.323 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:08:03.323 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:08:06.578 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:06.579 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:06.580 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:09.386 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T15:08:09.386 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:09.387 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:09.388 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:11.933 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T15:08:11.933 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:11.934 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:11.935 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:14.761 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T15:08:14.761 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:14.762 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '1a5c7941-36e1-49b6-b10c-5aec63eb825b', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:14.762 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 1a5c7941-36e1-49b6-b10c-5aec63eb825b --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:17.371 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T15:08:17.372 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:17.372 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:17.373 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:20.255 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T15:08:20.256 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:20.257 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:20.257 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:22.955 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T15:08:22.956 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:22.956 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:22.957 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:25.921 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T15:08:25.922 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:25.922 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'a91bfc36-ad1f-4fbc-8718-897c542ad285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:25.923 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period a91bfc36-ad1f-4fbc-8718-897c542ad285 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:28.601 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T15:08:28.602 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:28.603 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:28.603 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:31.391 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T15:08:31.392 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:31.395 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:31.402 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:33.977 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T15:08:33.978 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:33.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:33.979 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:36.971 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2020-05-24T15:08:36.972 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:36.973 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '9420dac7-f825-4d7d-a354-f516953dfe80', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:36.978 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 9420dac7-f825-4d7d-a354-f516953dfe80 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:39.570 INFO:teuthology.orchestra.run.smithi193.stdout:[] 2020-05-24T15:08:39.571 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_multi_period_incremental_sync ... ok 2020-05-24T15:08:39.573 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-46 2020-05-24T15:08:40.505 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-47 2020-05-24T15:08:42.087 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-48 2020-05-24T15:08:45.882 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:45.883 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:45.884 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:53.068 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332929.115796_141.1","last_update":"2020-05-24 15:08:49.115796Z"}] 2020-05-24T15:08:53.069 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:08:53.070 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:53.070 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:53.071 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:55.978 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17929869403099609634},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590332929.115796_141.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:49.115796Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:08:55.979 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:08:55.979 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:55.980 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:55.981 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:58.902 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332929.115796_141.1","last_update":"2020-05-24 15:08:49.115796Z"}] 2020-05-24T15:08:58.902 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:08:58.904 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:58.905 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:58.907 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:01.546 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":12405561517275143106},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590332929.115796_141.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:49.115796Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:01.547 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:09:01.548 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:01.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:01.549 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:04.227 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332929.115796_141.1","last_update":"2020-05-24 15:08:49.115796Z"}] 2020-05-24T15:09:04.228 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:09:04.228 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:04.229 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:04.230 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:07.648 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":8319721698348417080},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590332929.115796_141.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:49.115796Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:07.649 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:09:07.649 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:07.650 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:07.650 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:10.170 INFO:teuthology.orchestra.run.smithi193.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332928.413031_157.1","last_update":"2020-05-24 15:08:48.413031Z"}] 2020-05-24T15:09:10.170 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:09:10.171 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:10.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:10.172 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:13.262 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":5597757862415222515},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590332928.413031_157.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:48.413031Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:13.263 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:09:13.263 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:13.264 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:13.264 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:15.897 INFO:teuthology.orchestra.run.smithi193.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332928.413031_157.1","last_update":"2020-05-24 15:08:48.413031Z"}] 2020-05-24T15:09:15.897 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:09:15.898 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:15.899 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:15.899 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:18.819 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":9612765125548486980},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590332928.413031_157.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:48.413031Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:18.820 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:09:18.821 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:18.821 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:18.822 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:20.497 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:19:23.681 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:09:20.501 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:19:23.685 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:09:20.675 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:19:23.859 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:09:21.860 INFO:teuthology.orchestra.run.smithi193.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332928.413031_157.1","last_update":"2020-05-24 15:08:48.413031Z"}] 2020-05-24T15:09:21.860 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone2 2020-05-24T15:09:21.861 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:21.862 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:21.862 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:24.766 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:24.766 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:29.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:29.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:29.769 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:33.158 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:33.159 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:38.160 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:38.161 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:38.162 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:41.592 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:41.593 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:46.595 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:46.596 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:46.596 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:49.874 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:49.875 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:54.876 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:54.878 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:54.879 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:58.084 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:58.084 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:03.086 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:03.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:03.087 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:06.590 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:06.591 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:11.592 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:11.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:11.593 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:15.123 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:15.124 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:20.126 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:20.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:20.129 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:23.485 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:23.486 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:28.487 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:28.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:28.489 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:31.802 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:31.803 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:36.806 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:36.807 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:36.807 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:39.717 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:39.718 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:44.720 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:44.721 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:44.721 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:47.866 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:47.867 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:52.869 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:52.870 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:52.871 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:56.233 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:56.234 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:01.236 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:01.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:01.238 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:04.487 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:04.489 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:09.491 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:09.492 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:09.493 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:12.765 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:12.766 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:17.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:17.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:17.770 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:21.022 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:21.022 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:26.023 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:26.024 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:26.025 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:29.082 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:29.083 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:34.085 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:34.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:34.086 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:37.515 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:37.516 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:42.522 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:42.523 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:42.524 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:45.423 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:45.424 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:50.425 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:50.426 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:50.427 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:53.528 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:53.529 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:58.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:58.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:58.534 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:01.985 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:01.986 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:06.987 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:06.988 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:06.989 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:09.903 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:09.904 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:14.905 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:14.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:14.908 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:18.327 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:18.328 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:23.329 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:23.330 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:23.331 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:26.740 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:26.741 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:31.742 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:31.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:31.744 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:34.908 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:34.908 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:39.910 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:39.911 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:39.912 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:43.498 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:43.499 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:48.500 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:48.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:48.502 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:51.885 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:51.886 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:55.010 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:24:05.229 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:12:55.013 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:24:05.234 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:12:55.230 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:24:05.450 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:12:56.888 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:56.889 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:56.890 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:00.163 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:00.165 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:05.166 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:05.167 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:05.168 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:08.322 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:08.323 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:13.325 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:13.326 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:13.326 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:16.504 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:16.504 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:21.505 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:21.506 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:21.507 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:24.807 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:24.808 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:29.809 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:29.810 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:29.818 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:33.004 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:33.005 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:38.007 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:38.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:38.009 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:41.286 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:41.287 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:46.289 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:46.290 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:46.291 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:49.586 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:49.587 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:54.589 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:54.590 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:54.590 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:57.835 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:57.836 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:02.837 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:02.838 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:02.839 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:06.197 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:06.198 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:06.515 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:24:09.698 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:06.517 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:24:09.701 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:06.793 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:24:09.976 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:11.200 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:11.201 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:11.202 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:14.565 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:14.566 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:19.567 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:19.568 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:19.569 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:23.320 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:23.321 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:28.323 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:28.324 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:28.325 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:31.795 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:31.796 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:36.798 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:36.799 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:36.800 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:40.132 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:40.133 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:45.134 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:45.135 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:45.136 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:48.088 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:48.089 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:53.090 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:53.092 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:53.092 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:56.608 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:56.609 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:57.384 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:26:07.604 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:57.399 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:26:07.619 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:57.539 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:26:07.759 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:15:01.613 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:01.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:01.614 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:04.694 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:04.695 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:09.697 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:09.698 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:09.698 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:12.906 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:12.906 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:17.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:17.909 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:17.910 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:21.072 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:21.072 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:26.074 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:26.075 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:26.076 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:29.580 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:29.581 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:34.583 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:34.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:34.584 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:38.314 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:38.315 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:43.316 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:43.317 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:43.318 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:46.965 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:46.966 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:51.967 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:51.968 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:51.969 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:55.635 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:55.636 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:00.638 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:00.639 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:00.640 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:03.877 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:03.878 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:08.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:08.886 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:08.887 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:12.408 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:12.409 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:16.202 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:26:19.385 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:16:16.232 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:26:19.416 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:16:16.430 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:26:19.613 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:16:17.410 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:17.410 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:17.411 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:21.100 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:21.101 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:26.103 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:26.104 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:26.105 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:29.275 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:29.276 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:34.277 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:34.278 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:34.279 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:37.795 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:37.796 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:42.798 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:42.799 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:42.799 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:46.177 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:46.178 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:51.179 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:51.180 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:51.181 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:54.167 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:54.167 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:59.169 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:59.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:59.171 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:17:02.486 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:17:02.486 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:17:07.488 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:07.489 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:07.490 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:17:10.497 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:17:10.498 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:17:15.500 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:15.500 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:15.501 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:17:18.547 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:17:18.548 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:17:23.551 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:23.552 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:23.553 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:17:27.238 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:17:27.239 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:17:32.241 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:32.242 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:32.243 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:17:35.753 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:17:35.754 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:17:40.763 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_datalog_autotrim ... FAIL 2020-05-24T15:17:40.764 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:40.765 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:40.766 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=false --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:17:41.342 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:17:41.338 7fba6652e840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T15:17:41.343 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:17:41.338 7fba6652e840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T15:17:41.411 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:17:41.409 7fba6652e840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T15:17:41.411 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:17:41.409 7fba6652e840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T15:17:41.623 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:17:41.623 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:41.624 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:41.625 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:17:44.180 INFO:teuthology.orchestra.run.smithi193.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T15:17:48.401 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":2,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"false","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:17:48.402 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to false 2020-05-24T15:18:48.404 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-49 2020-05-24T15:18:49.752 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:49.753 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:49.755 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:51.809 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=9420dac7-f825-4d7d-a354-f516953dfe80 2020-05-24T15:18:52.491 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590333529.190298_137.1","last_update":"2020-05-24 15:18:49.190298Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:18:52.492 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:18:52.493 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:52.494 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:52.494 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:55.089 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590333529.190298_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:49.190298Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:18:55.089 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:18:55.090 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:18:55.090 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:55.091 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:55.092 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:58.087 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332923.310646_110.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:43.310646Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:18:58.087 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590333529.190298_137.1 target=1_1590332923.310646_110.1 2020-05-24T15:19:03.089 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:03.090 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:03.091 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:06.359 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332923.310646_110.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:43.310646Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:06.360 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590333529.190298_137.1 target=1_1590332923.310646_110.1 2020-05-24T15:19:11.362 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:11.363 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:11.364 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:14.450 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332923.310646_110.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:43.310646Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:14.451 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590333529.190298_137.1 target=1_1590332923.310646_110.1 2020-05-24T15:19:19.452 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:19.453 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:19.454 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:22.310 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332923.310646_110.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:43.310646Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:22.311 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590333529.190298_137.1 target=1_1590332923.310646_110.1 2020-05-24T15:19:27.319 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:27.320 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:27.321 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:30.476 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332923.310646_110.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:43.310646Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:30.477 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590333529.190298_137.1 target=1_1590332923.310646_110.1 2020-05-24T15:19:35.478 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:35.479 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:35.480 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:38.400 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332923.310646_110.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:43.310646Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:38.402 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590333529.190298_137.1 target=1_1590332923.310646_110.1 2020-05-24T15:19:43.403 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:43.405 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:43.406 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:46.372 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590332923.310646_110.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:43.310646Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:46.373 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590333529.190298_137.1 target=1_1590332923.310646_110.1 2020-05-24T15:19:51.374 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:51.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:51.375 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:55.446 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590333529.190298_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:49.190298Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:55.447 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:19:55.448 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:19:55.448 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:55.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:55.449 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:58.628 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590333529.190298_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:49.190298Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:58.629 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:19:58.998 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=4e214d8a-46aa-488b-8afd-3016515a7228', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:58.999 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=4e214d8a-46aa-488b-8afd-3016515a7228', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:59.000 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone=4e214d8a-46aa-488b-8afd-3016515a7228 --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:19:59.417 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:19:59.414 7fe77b986840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T15:19:59.418 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:19:59.414 7fe77b986840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T15:19:59.464 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:19:59.461 7fe77b986840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T15:19:59.464 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:19:59.461 7fe77b986840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T15:19:59.608 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:19:59.609 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:59.610 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:59.610 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:20:02.652 INFO:teuthology.orchestra.run.smithi193.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T15:20:07.069 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":3,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"false","sync_from":[],"redirect_zone":"4e214d8a-46aa-488b-8afd-3016515a7228"},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:20:07.071 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "4e214d8a-46aa-488b-8afd-3016515a7228" 2020-05-24T15:21:10.859 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:10.860 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:10.862 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=true --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:21:11.353 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:21:11.349 7f4b9292e840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T15:21:11.353 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:21:11.349 7f4b9292e840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T15:21:11.402 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:21:11.397 7f4b9292e840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T15:21:11.403 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:21:11.397 7f4b9292e840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T15:21:11.509 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:21:11.511 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:11.512 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:11.512 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:21:14.349 INFO:teuthology.orchestra.run.smithi193.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T15:21:15.697 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:32:25.907 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:15.698 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:32:25.910 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:15.910 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:32:26.130 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:17.786 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":4,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":"4e214d8a-46aa-488b-8afd-3016515a7228"},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:21:17.787 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to true 2020-05-24T15:21:51.700 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:33:01.918 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:51.703 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:33:01.923 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:51.762 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:33:01.982 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:22:17.798 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:22:17.799 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:22:17.800 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone= --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:22:18.367 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:22:18.363 7f82e01b0840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T15:22:18.368 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:22:18.363 7f82e01b0840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T15:22:18.420 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:22:18.417 7f82e01b0840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T15:22:18.421 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:22:18.417 7f82e01b0840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T15:22:18.578 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:22:18.579 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:22:18.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:22:18.582 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:22:21.399 INFO:teuthology.orchestra.run.smithi193.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T15:22:25.506 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":5,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:22:25.507 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "" 2020-05-24T15:22:27.563 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:32:30.747 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:22:27.568 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:32:30.753 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:22:27.729 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:32:30.913 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:23:11.263 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:33:14.445 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:23:11.271 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:33:14.455 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:23:11.389 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:33:14.573 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:23:25.509 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_multi_zone_redirect ... ok 2020-05-24T15:23:25.511 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:25.512 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:25.513 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:23:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=9420dac7-f825-4d7d-a354-f516953dfe80 2020-05-24T15:23:28.819 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590333529.190298_137.1","last_update":"2020-05-24 15:18:49.190298Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:23:28.820 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:23:28.820 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:28.821 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:28.821 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:23:32.040 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590333529.190298_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:49.190298Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:23:32.041 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:23:32.041 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:23:32.051 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:32.051 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:32.052 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:23:34.884 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590333529.190298_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:49.190298Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:23:34.885 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:23:34.885 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:23:34.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:34.886 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:34.887 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:23:38.479 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590333529.190298_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:49.190298Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:23:38.480 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:23:38.480 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:38.481 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '3VNXH5BI34IE2RAR', '--secret', 'zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:38.482 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key 3VNXH5BI34IE2RAR --secret zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:23:39.667 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:23:39.664 7fd9eec10840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T15:23:39.668 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:23:39.664 7fd9eec10840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T15:23:39.732 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:23:39.729 7fd9eec10840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T15:23:39.733 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:23:39.729 7fd9eec10840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T15:23:39.810 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"2c7679bc-40dc-49cc-9aef-e61e0b36f700","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","system_key":{"access_key":"3VNXH5BI34IE2RAR","secret_key":"zhaySPUezXyL2v4FpsJqI8o6c5ejMtEO"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:23:39.811 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:39.812 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:39.813 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:17.261 INFO:teuthology.orchestra.run.smithi193.stdout:Sending period to new master zone 4e214d8a-46aa-488b-8afd-3016515a7228 2020-05-24T15:24:21.074 INFO:teuthology.orchestra.run.smithi193.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":6,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"2c7679bc-40dc-49cc-9aef-e61e0b36f700","name":"remove","endpoints":[],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"2c7679bc-40dc-49cc-9aef-e61e0b36f700","val":3444587726},{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:24:21.075 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:21.076 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:21.077 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:21.523 7f0e7f03f840 0 failed reading obj info from .rgw.root:zone_info.692d187d-2dfb-4f37-a732-96d22188c425: (2) No such file or directory 2020-05-24T15:24:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:21.523 7f0e7f03f840 0 WARNING: could not read zone params for zone id=692d187d-2dfb-4f37-a732-96d22188c425 name=test-zone2 2020-05-24T15:24:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:21.557 7f0e7f03f840 0 failed reading obj info from .rgw.root:zone_info.7f588a25-0d83-455c-8d94-e1a778babeb4: (2) No such file or directory 2020-05-24T15:24:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:21.557 7f0e7f03f840 0 WARNING: could not read zone params for zone id=7f588a25-0d83-455c-8d94-e1a778babeb4 name=test-zone4 2020-05-24T15:24:21.676 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:24:21.677 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:21.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:21.678 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:22.071 INFO:teuthology.orchestra.run.smithi019.stderr:zone name remove not found in zonegroup test-zonegroup 2020-05-24T15:24:22.076 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-05-24T15:24:22.077 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:22.077 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:22.078 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone delete --rgw-zone remove --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:23.211 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:24:23.208 7f414f8f6840 0 failed reading obj info from .rgw.root:zone_info.4e214d8a-46aa-488b-8afd-3016515a7228: (2) No such file or directory 2020-05-24T15:24:23.212 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:24:23.208 7f414f8f6840 0 WARNING: could not read zone params for zone id=4e214d8a-46aa-488b-8afd-3016515a7228 name=test-zone1 2020-05-24T15:24:23.254 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:24:23.251 7f414f8f6840 0 failed reading obj info from .rgw.root:zone_info.b1afa591-9013-48bc-9214-4885d16784dd: (2) No such file or directory 2020-05-24T15:24:23.255 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:24:23.251 7f414f8f6840 0 WARNING: could not read zone params for zone id=b1afa591-9013-48bc-9214-4885d16784dd name=test-zone3 2020-05-24T15:24:23.351 INFO:teuthology.orchestra.run.smithi193.stderr:2020-05-24 15:24:23.348 7f414f8f6840 0 zone id 2c7679bc-40dc-49cc-9aef-e61e0b36f700 is not a part of zonegroup default 2020-05-24T15:24:23.570 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:23.571 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:23.572 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:26.552 7f54d519c840 0 WARNING: can't generate connection for zone 2c7679bc-40dc-49cc-9aef-e61e0b36f700 id remove: no endpoints defined 2020-05-24T15:24:33.666 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":7,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:24:33.668 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_zonegroup_remove ... ok 2020-05-24T15:24:33.669 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:33.670 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:33.671 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup create --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:34.367 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"efb6079c-fde2-4e22-a85c-76e087252b2e","name":"remove_zg","api_name":"remove_zg","is_master":"false","endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[],"default_placement":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:24:34.368 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:34.369 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:34.370 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:36.658 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"7b83ff34-d65f-4383-bd8f-5d48f68f0844","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"} 2020-05-24T15:24:36.659 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:36.660 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:36.661 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:42.412 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":8,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"efb6079c-fde2-4e22-a85c-76e087252b2e","name":"remove_zg","api_name":"remove_zg","is_master":"false","endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"7b83ff34-d65f-4383-bd8f-5d48f68f0844","zones":[{"id":"7b83ff34-d65f-4383-bd8f-5d48f68f0844","name":"remove","endpoints":[],"log_meta":"false","log_data":"false","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"},{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7b83ff34-d65f-4383-bd8f-5d48f68f0844","val":3326479426},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:24:42.413 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:42.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:42.415 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone delete --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:43.700 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:43.697 7f331c1e9840 0 zone id 7b83ff34-d65f-4383-bd8f-5d48f68f0844 is not a part of zonegroup test-zonegroup 2020-05-24T15:24:43.749 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:43.744 7f331c1e9840 0 zone id 7b83ff34-d65f-4383-bd8f-5d48f68f0844 is not a part of zonegroup default 2020-05-24T15:24:43.951 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:43.952 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:43.953 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2020-05-24 15:24:45.145 7ffa1ca16840 0 ERROR: zonegroup remove_zg has a non existent master zone 2020-05-24T15:24:45.153 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-05-24T15:24:45.154 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:45.155 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:45.156 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup delete --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:45.777 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:45.778 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:45.779 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:51.551 INFO:teuthology.orchestra.run.smithi019.stdout:{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","epoch":9,"predecessor_uuid":"a91bfc36-ad1f-4fbc-8718-897c542ad285","sync_status":["1_1590332710.970452_141.1","","",""],"period_map":{"id":"9420dac7-f825-4d7d-a354-f516953dfe80","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","zones":[{"id":"4e214d8a-46aa-488b-8afd-3016515a7228","name":"test-zone1","endpoints":["http://smithi019.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"692d187d-2dfb-4f37-a732-96d22188c425","name":"test-zone2","endpoints":["http://smithi193.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7f588a25-0d83-455c-8d94-e1a778babeb4","name":"test-zone4","endpoints":["http://smithi193.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"b1afa591-9013-48bc-9214-4885d16784dd","name":"test-zone3","endpoints":["http://smithi019.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858"}],"short_zone_ids":[{"key":"4e214d8a-46aa-488b-8afd-3016515a7228","val":663465406},{"key":"692d187d-2dfb-4f37-a732-96d22188c425","val":2806809540},{"key":"7f588a25-0d83-455c-8d94-e1a778babeb4","val":2029961734},{"key":"b1afa591-9013-48bc-9214-4885d16784dd","val":2007059419}]},"master_zonegroup":"test-zonegroup","master_zone":"4e214d8a-46aa-488b-8afd-3016515a7228","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"4aca72c0-4e47-41a6-ae5c-598b4dc05858","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:24:51.552 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_zg_master_zone_delete ... ok 2020-05-24T15:24:51.553 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-50 2020-05-24T15:25:03.375 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-51 2020-05-24T15:25:05.617 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-52 2020-05-24T15:25:08.722 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-53 2020-05-24T15:25:10.495 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-54 2020-05-24T15:25:30.283 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-55 2020-05-24T15:25:37.406 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_set_bucket_website ... tasks.rgw_multi.tests.test_set_bucket_policy ... ok 2020-05-24T15:25:37.407 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-56 2020-05-24T15:25:38.228 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-57 2020-05-24T15:25:39.904 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-58 2020-05-24T15:25:41.454 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'fwzbiz-56', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:41.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'fwzbiz-56', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:41.455 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket fwzbiz-56 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:25:44.781 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'fwzbiz-57', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:44.782 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'fwzbiz-57', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:44.783 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket fwzbiz-57 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:25:48.049 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'fwzbiz-58', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:48.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'fwzbiz-58', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:48.051 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket fwzbiz-58 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:25:51.490 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:51.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:51.492 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:54.720 INFO:teuthology.orchestra.run.smithi019.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.2-fwzbiz-45","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","obj_delete_at_hint.0000000028","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.47-fwzbiz-38","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.2","obj_delete_at_hint.0000000040","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","meta.full-sync.index.0","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.2-fwzbiz-51","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.37-fwzbiz-28","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","datalog.sync-status.b1afa591-9013-48bc-9214-4885d16784dd","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-48:4e214d8a-46aa-488b-8afd-3016515a7228.5043.5","data_log.3","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.5-fwzbiz-5","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.42-fwzbiz-33","obj_delete_at_hint.0000000076","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.30-fwzbiz-21","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.39-fwzbiz-30","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.1-fwzbiz-50","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.1-fwzbiz-1","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-48:4e214d8a-46aa-488b-8afd-3016515a7228.5043.5","obj_delete_at_hint.0000000089","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.19-fwzbiz-10","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.3.retry","obj_delete_at_hint.0000000022","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.23-fwzbiz-14","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.3-fwzbiz-3","bilog.trim","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.38-fwzbiz-29","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.43-fwzbiz-34","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.2-fwzbiz-2","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.0","obj_delete_at_hint.0000000074","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-47:4e214d8a-46aa-488b-8afd-3016515a7228.5043.4","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.33-fwzbiz-24","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.1","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.41-fwzbiz-32","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.3","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","meta.full-sync.index.1","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.1-fwzbiz-44","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.27-fwzbiz-18","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.5-fwzbiz-48","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.4-fwzbiz-53","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.3","obj_delete_at_hint.0000000053","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.5-fwzbiz-54","sync.error-log.3","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-46:4e214d8a-46aa-488b-8afd-3016515a7228.5043.3","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.40-fwzbiz-31","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.6-fwzbiz-6","meta.log.9420dac7-f825-4d7d-a354-f516953dfe80.0","mdlog.sync-status.shard.2","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-47:4e214d8a-46aa-488b-8afd-3016515a7228.5043.4","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.32-fwzbiz-23","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-49:4e214d8a-46aa-488b-8afd-3016515a7228.5228.1","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","mdlog.sync-status.shard.1","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.6-fwzbiz-55","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.17-fwzbiz-8","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.3","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.9-fwzbiz-58","obj_delete_at_hint.0000000066","2020-05-24-15-692d187d-2dfb-4f37-a732-96d22188c425.8106.1-fwzbiz-42","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.34-fwzbiz-25","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.4-fwzbiz-4","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.44-fwzbiz-35","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","mdlog.sync-status.shard.0","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.0","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.1","obj_delete_at_hint.0000000039","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.36-fwzbiz-27","obj_delete_at_hint.0000000100","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.3.retry","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.3-fwzbiz-52","obj_delete_at_hint.0000000093","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.20-fwzbiz-11","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-46:4e214d8a-46aa-488b-8afd-3016515a7228.5043.3","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","sync.error-log.2","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.35-fwzbiz-26","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.1.retry","obj_delete_at_hint.0000000012","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.0","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.22-fwzbiz-13","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.1","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.1","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-49:4e214d8a-46aa-488b-8afd-3016515a7228.5228.1","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.49-fwzbiz-40","obj_delete_at_hint.0000000020","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.46-fwzbiz-37","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.21-fwzbiz-12","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.3","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.48-fwzbiz-39","obj_delete_at_hint.0000000115","sync.error-log.1","obj_delete_at_hint.0000000033","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5228.1-fwzbiz-49","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.8-fwzbiz-57","obj_delete_at_hint.0000000025","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","obj_delete_at_hint.0000000123","datalog.sync-status.692d187d-2dfb-4f37-a732-96d22188c425","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.31-fwzbiz-22","obj_delete_at_hint.0000000013","2020-05-24-15-692d187d-2dfb-4f37-a732-96d22188c425.8106.2-fwzbiz-43","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.16-fwzbiz-7","obj_delete_at_hint.0000000051","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.50-fwzbiz-41","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.4-fwzbiz-47","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.18-fwzbiz-9","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.25-fwzbiz-16","obj_delete_at_hint.0000000046","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","meta.full-sync.index.2","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.28-fwzbiz-19","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.0","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.29-fwzbiz-20","obj_delete_at_hint.0000000067","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.2","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.18-fwzbiz-9","obj_delete_at_hint.0000000086","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.2","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.7-fwzbiz-56","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.26-fwzbiz-17","obj_delete_at_hint.0000000050","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","obj_delete_at_hint.0000000102","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.24-fwzbiz-15","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.45-fwzbiz-36","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.3-fwzbiz-46","obj_delete_at_hint.0000000097","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.2.retry","obj_delete_at_hint.0000000042","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.2","obj_delete_at_hint.0000000075","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2020-05-24T15:25:55.329 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:55.331 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:55.332 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:58.925 INFO:teuthology.orchestra.run.smithi193.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.3","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.2-fwzbiz-45","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","obj_delete_at_hint.0000000028","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.47-fwzbiz-38","obj_delete_at_hint.0000000040","meta.full-sync.index.0","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.2-fwzbiz-51","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.37-fwzbiz-28","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","datalog.sync-status.b1afa591-9013-48bc-9214-4885d16784dd","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-48:4e214d8a-46aa-488b-8afd-3016515a7228.5043.5","data_log.3","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.2","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.5-fwzbiz-5","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.1","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.30-fwzbiz-21","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.1-fwzbiz-1","obj_delete_at_hint.0000000096","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.19-fwzbiz-10","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.3.retry","obj_delete_at_hint.0000000022","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.23-fwzbiz-14","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.3-fwzbiz-3","bilog.trim","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-49:4e214d8a-46aa-488b-8afd-3016515a7228.5228.1","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.43-fwzbiz-34","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.2-fwzbiz-2","obj_delete_at_hint.0000000064","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-46:4e214d8a-46aa-488b-8afd-3016515a7228.5043.3","obj_delete_at_hint.0000000071","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.0","obj_delete_at_hint.0000000074","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.33-fwzbiz-24","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.1","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","meta.full-sync.index.1","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.1-fwzbiz-44","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-47:4e214d8a-46aa-488b-8afd-3016515a7228.5043.4","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.5-fwzbiz-48","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-48:4e214d8a-46aa-488b-8afd-3016515a7228.5043.5","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.3","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.5-fwzbiz-54","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","sync.error-log.3","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.40-fwzbiz-31","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.6-fwzbiz-6","meta.log.9420dac7-f825-4d7d-a354-f516953dfe80.0","mdlog.sync-status.shard.2","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-47:4e214d8a-46aa-488b-8afd-3016515a7228.5043.4","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.32-fwzbiz-23","obj_delete_at_hint.0000000116","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-49:4e214d8a-46aa-488b-8afd-3016515a7228.5228.1","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.2","mdlog.sync-status.shard.1","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.17-fwzbiz-8","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.0.retry","2020-05-24-15-692d187d-2dfb-4f37-a732-96d22188c425.8106.1-fwzbiz-42","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.3","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.34-fwzbiz-25","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.4-fwzbiz-4","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","mdlog.sync-status.shard.0","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.3.retry","obj_delete_at_hint.0000000093","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.20-fwzbiz-11","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-46:4e214d8a-46aa-488b-8afd-3016515a7228.5043.3","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","sync.error-log.2","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.35-fwzbiz-26","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.1.retry","obj_delete_at_hint.0000000012","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.1","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.22-fwzbiz-13","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.1","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.49-fwzbiz-40","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","obj_delete_at_hint.0000000020","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.46-fwzbiz-37","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.21-fwzbiz-12","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","obj_delete_at_hint.0000000072","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","obj_delete_at_hint.0000000062","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.3","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.48-fwzbiz-39","obj_delete_at_hint.0000000115","sync.error-log.1","obj_delete_at_hint.0000000033","datalog.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5228.1-fwzbiz-49","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.8-fwzbiz-57","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.31-fwzbiz-22","obj_delete_at_hint.0000000013","2020-05-24-15-692d187d-2dfb-4f37-a732-96d22188c425.8106.2-fwzbiz-43","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.16-fwzbiz-7","obj_delete_at_hint.0000000051","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.50-fwzbiz-41","obj_delete_at_hint.0000000106","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.0","obj_delete_at_hint.0000000017","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.4-fwzbiz-47","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.25-fwzbiz-16","obj_delete_at_hint.0000000046","meta.full-sync.index.2","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.28-fwzbiz-19","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.0","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.29-fwzbiz-20","obj_delete_at_hint.0000000067","data.full-sync.index.b1afa591-9013-48bc-9214-4885d16784dd.2","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.18-fwzbiz-9","obj_delete_at_hint.0000000086","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.0","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.26-fwzbiz-17","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","obj_delete_at_hint.0000000124","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.24-fwzbiz-15","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.45-fwzbiz-36","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.3-fwzbiz-46","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","obj_delete_at_hint.0000000097","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.2.retry","obj_delete_at_hint.0000000042","datalog.sync-status.shard.b1afa591-9013-48bc-9214-4885d16784dd.2","obj_delete_at_hint.0000000075","bucket.sync-status.b1afa591-9013-48bc-9214-4885d16784dd:fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2020-05-24T15:25:59.613 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:59.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:59.615 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:03.155 INFO:teuthology.orchestra.run.smithi019.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.3","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.2-fwzbiz-45","obj_delete_at_hint.0000000028","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.47-fwzbiz-38","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.2","obj_delete_at_hint.0000000040","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","meta.full-sync.index.0","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","data_log.3","meta.log.a91bfc36-ad1f-4fbc-8718-897c542ad285.0","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.2","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.5-fwzbiz-5","obj_delete_at_hint.0000000076","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.1","meta.log.1a5c7941-36e1-49b6-b10c-5aec63eb825b.1","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.30-fwzbiz-21","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.1-fwzbiz-1","obj_delete_at_hint.0000000096","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-48:4e214d8a-46aa-488b-8afd-3016515a7228.5043.5","obj_delete_at_hint.0000000089","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.19-fwzbiz-10","obj_delete_at_hint.0000000022","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.23-fwzbiz-14","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.3-fwzbiz-3","bilog.trim","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.38-fwzbiz-29","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-49:4e214d8a-46aa-488b-8afd-3016515a7228.5228.1","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.2-fwzbiz-2","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","obj_delete_at_hint.0000000064","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-46:4e214d8a-46aa-488b-8afd-3016515a7228.5043.3","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-47:4e214d8a-46aa-488b-8afd-3016515a7228.5043.4","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.33-fwzbiz-24","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.41-fwzbiz-32","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.3","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","meta.full-sync.index.1","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.1-fwzbiz-44","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-47:4e214d8a-46aa-488b-8afd-3016515a7228.5043.4","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.27-fwzbiz-18","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.5-fwzbiz-48","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-48:4e214d8a-46aa-488b-8afd-3016515a7228.5043.5","obj_delete_at_hint.0000000053","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-46:4e214d8a-46aa-488b-8afd-3016515a7228.5043.3","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.6-fwzbiz-6","meta.log.9420dac7-f825-4d7d-a354-f516953dfe80.0","mdlog.sync-status.shard.2","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.32-fwzbiz-23","obj_delete_at_hint.0000000116","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.2","mdlog.sync-status.shard.1","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.6-fwzbiz-55","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.17-fwzbiz-8","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.3","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.9-fwzbiz-58","obj_delete_at_hint.0000000066","2020-05-24-15-692d187d-2dfb-4f37-a732-96d22188c425.8106.1-fwzbiz-42","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.3","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.34-fwzbiz-25","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.4-fwzbiz-4","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.44-fwzbiz-35","mdlog.sync-status.shard.0","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.0","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.1","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.3.retry","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.3.retry","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5454.3-fwzbiz-52","obj_delete_at_hint.0000000093","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.20-fwzbiz-11","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.35-fwzbiz-26","obj_delete_at_hint.0000000012","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.0","meta.log.1a5c7941-36e1-49b6-b10c-5aec63eb825b.0","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.1","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.22-fwzbiz-13","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","data.full-sync.index.692d187d-2dfb-4f37-a732-96d22188c425.1","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-49:4e214d8a-46aa-488b-8afd-3016515a7228.5228.1","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.49-fwzbiz-40","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","obj_delete_at_hint.0000000020","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.46-fwzbiz-37","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.21-fwzbiz-12","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","obj_delete_at_hint.0000000072","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","obj_delete_at_hint.0000000062","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.48-fwzbiz-39","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","datalog.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5228.1-fwzbiz-49","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","obj_delete_at_hint.0000000025","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","obj_delete_at_hint.0000000123","datalog.sync-status.692d187d-2dfb-4f37-a732-96d22188c425","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.31-fwzbiz-22","obj_delete_at_hint.0000000013","2020-05-24-15-692d187d-2dfb-4f37-a732-96d22188c425.8106.2-fwzbiz-43","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.16-fwzbiz-7","obj_delete_at_hint.0000000051","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.50-fwzbiz-41","obj_delete_at_hint.0000000106","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.0","obj_delete_at_hint.0000000017","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.4-fwzbiz-47","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.18-fwzbiz-9","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.25-fwzbiz-16","obj_delete_at_hint.0000000046","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","meta.full-sync.index.2","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.28-fwzbiz-19","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.29-fwzbiz-20","obj_delete_at_hint.0000000067","2020-05-24-13-4e214d8a-46aa-488b-8afd-3016515a7228.4387.18-fwzbiz-9","obj_delete_at_hint.0000000086","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","datalog.sync-status.shard.692d187d-2dfb-4f37-a732-96d22188c425.2","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.0","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.26-fwzbiz-17","obj_delete_at_hint.0000000050","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","bucket.sync-status.692d187d-2dfb-4f37-a732-96d22188c425:fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","2020-05-24-14-4e214d8a-46aa-488b-8afd-3016515a7228.4387.24-fwzbiz-15","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.4387.45-fwzbiz-36","2020-05-24-15-4e214d8a-46aa-488b-8afd-3016515a7228.5043.3-fwzbiz-46","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2020-05-24T15:26:04.242 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone4', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:04.243 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone4', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:04.243 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone4 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:11.072 INFO:teuthology.orchestra.run.smithi193.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.3","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","meta.full-sync.index.0","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-22:4e214d8a-46aa-488b-8afd-3016515a7228.4387.31","meta.log.a91bfc36-ad1f-4fbc-8718-897c542ad285.0","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-11:4e214d8a-46aa-488b-8afd-3016515a7228.4387.20","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.2","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.1","meta.log.1a5c7941-36e1-49b6-b10c-5aec63eb825b.1","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-17:4e214d8a-46aa-488b-8afd-3016515a7228.4387.26","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-37:4e214d8a-46aa-488b-8afd-3016515a7228.4387.46","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","bilog.trim","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-26:4e214d8a-46aa-488b-8afd-3016515a7228.4387.35","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-36:4e214d8a-46aa-488b-8afd-3016515a7228.4387.45","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-49:4e214d8a-46aa-488b-8afd-3016515a7228.5228.1","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-46:4e214d8a-46aa-488b-8afd-3016515a7228.5043.3","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-23:4e214d8a-46aa-488b-8afd-3016515a7228.4387.32","meta.full-sync.index.1","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-47:4e214d8a-46aa-488b-8afd-3016515a7228.5043.4","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-48:4e214d8a-46aa-488b-8afd-3016515a7228.5043.5","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-25:4e214d8a-46aa-488b-8afd-3016515a7228.4387.34","meta.log.9420dac7-f825-4d7d-a354-f516953dfe80.0","mdlog.sync-status.shard.2","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-24:4e214d8a-46aa-488b-8afd-3016515a7228.4387.33","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.2","mdlog.sync-status.shard.1","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-10:4e214d8a-46aa-488b-8afd-3016515a7228.4387.19","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-16:4e214d8a-46aa-488b-8afd-3016515a7228.4387.25","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.3","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","mdlog.sync-status.shard.0","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.3.retry","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","obj_delete_at_hint.0000000012","meta.log.1a5c7941-36e1-49b6-b10c-5aec63eb825b.0","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.1","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-13:4e214d8a-46aa-488b-8afd-3016515a7228.4387.22","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-21:4e214d8a-46aa-488b-8afd-3016515a7228.4387.30","obj_delete_at_hint.0000000072","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-15:4e214d8a-46aa-488b-8afd-3016515a7228.4387.24","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","datalog.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-14:4e214d8a-46aa-488b-8afd-3016515a7228.4387.23","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","data.full-sync.index.4e214d8a-46aa-488b-8afd-3016515a7228.0","obj_delete_at_hint.0000000017","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-19:4e214d8a-46aa-488b-8afd-3016515a7228.4387.28","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","meta.full-sync.index.2","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-12:4e214d8a-46aa-488b-8afd-3016515a7228.4387.21","datalog.sync-status.shard.4e214d8a-46aa-488b-8afd-3016515a7228.0","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-38:4e214d8a-46aa-488b-8afd-3016515a7228.4387.47","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","bucket.sync-status.4e214d8a-46aa-488b-8afd-3016515a7228:fwzbiz-20:4e214d8a-46aa-488b-8afd-3016515a7228.4387.29","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2020-05-24T15:26:12.048 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:12.049 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:12.050 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:15.209 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333950.946898_211.1","last_update":"2020-05-24 15:25:50.946898Z"}] 2020-05-24T15:26:15.210 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:26:15.210 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:15.211 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:15.212 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:17.818 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17929869403099609634},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590333674.281525_173.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:21:14.281525Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:17.819 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=3 master=1_1590333950.946898_211.1 target=1_1590333674.281525_173.1 2020-05-24T15:26:22.820 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:22.821 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:22.823 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:25.701 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17929869403099609634},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590333674.281525_173.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:21:14.281525Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:25.702 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=3 master=1_1590333950.946898_211.1 target=1_1590333674.281525_173.1 2020-05-24T15:26:30.706 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:30.708 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:30.709 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:33.394 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17929869403099609634},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590333950.946898_211.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:50.946898Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:33.395 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:26:33.396 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:33.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:33.397 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:36.213 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333950.946898_211.1","last_update":"2020-05-24 15:25:50.946898Z"}] 2020-05-24T15:26:36.214 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:26:36.215 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:36.215 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:36.216 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:38.841 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":12405561517275143106},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590333950.946898_211.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:50.946898Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:38.841 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:26:38.842 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:38.843 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:38.843 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:41.530 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333950.946898_211.1","last_update":"2020-05-24 15:25:50.946898Z"}] 2020-05-24T15:26:41.530 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:26:41.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:41.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:41.532 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:45.023 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":8319721698348417080},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590333950.946898_211.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:50.946898Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:45.024 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:26:45.024 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:45.025 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:45.025 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:47.638 INFO:teuthology.orchestra.run.smithi193.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333971.738872_233.1","last_update":"2020-05-24 15:26:11.738872Z"}] 2020-05-24T15:26:47.639 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:26:47.640 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:47.641 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:47.642 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:50.651 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":5597757862415222515},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590333971.738872_233.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:26:11.738872Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:50.652 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:26:50.653 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:50.653 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:50.654 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:53.169 INFO:teuthology.orchestra.run.smithi193.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333971.738872_233.1","last_update":"2020-05-24 15:26:11.738872Z"}] 2020-05-24T15:26:53.170 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:26:53.170 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:53.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:53.172 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:56.169 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":9612765125548486980},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1590333971.738872_233.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:26:11.738872Z"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:56.170 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:26:56.171 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:56.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:56.172 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:58.837 INFO:teuthology.orchestra.run.smithi193.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333971.738872_233.1","last_update":"2020-05-24 15:26:11.738872Z"}] 2020-05-24T15:26:58.838 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone2 2020-05-24T15:26:58.838 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:58.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:58.840 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:02.450 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:02.451 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:07.461 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:07.466 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:07.468 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:10.674 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:10.675 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:15.677 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:15.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:15.679 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:19.078 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:19.079 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:24.080 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:24.081 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:24.081 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:27.437 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:27.437 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:32.439 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:32.440 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:32.441 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:35.346 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:35.347 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:40.349 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:40.350 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:40.350 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:43.699 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:43.700 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:48.702 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:48.703 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:48.704 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:51.937 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:51.937 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:56.939 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:56.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:56.940 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:00.413 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:00.413 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:05.415 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:05.416 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:05.417 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:08.924 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:08.924 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:13.926 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:13.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:13.927 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:17.207 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:17.208 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:22.217 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:22.218 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:22.219 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:25.466 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:25.466 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:30.467 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:30.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:30.469 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:33.899 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:33.900 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:38.902 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:38.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:38.904 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:42.361 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:42.362 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:47.363 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:47.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:47.365 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:50.329 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:40:00.548 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:50.339 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:40:00.559 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:50.384 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:50.385 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:50.723 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:40:00.943 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:55.387 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:55.388 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:55.389 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:58.691 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:58.691 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:03.693 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:03.694 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:03.695 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:06.938 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:06.939 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:11.940 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:11.941 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:11.941 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:15.141 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:15.141 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:20.144 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:20.145 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:20.145 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:21.505 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:39:24.689 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:21.515 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:39:24.699 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:21.838 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:39:25.021 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:23.287 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:23.288 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:28.289 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:28.290 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:28.291 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:31.867 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:31.868 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:36.609 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:40:46.828 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:36.610 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:40:46.831 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:36.660 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:40:46.880 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:36.869 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:36.870 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:36.870 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:40.052 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:40.053 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:45.054 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:45.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:45.056 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:48.289 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:48.290 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:53.291 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:53.292 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:53.293 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:56.643 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:56.644 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:01.646 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:01.647 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:01.648 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:05.142 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:05.143 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:10.145 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:10.147 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:10.148 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:13.578 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:13.579 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:18.580 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:18.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:18.582 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:22.101 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:22.102 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:27.103 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:27.104 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:27.105 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:30.589 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:30.589 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:35.591 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:35.592 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:35.597 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:39.054 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:39.055 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:44.056 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:44.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:44.058 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:47.335 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:47.336 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:48.359 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:40:51.541 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:30:48.398 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:40:51.582 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:30:48.468 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:40:51.652 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:30:52.338 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:52.339 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:52.339 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:55.629 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:55.630 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:00.631 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:00.632 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:00.633 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:03.842 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:03.842 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:08.843 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:08.844 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:08.845 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:12.025 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:12.025 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:17.027 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:17.028 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:17.029 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:20.286 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:20.287 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:25.289 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:25.290 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:25.291 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:29.164 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:29.164 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:34.165 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:34.167 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:34.167 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:37.776 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:37.777 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:42.779 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:42.779 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:42.780 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:46.402 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:46.403 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:51.404 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:51.405 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:51.406 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:54.686 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:54.687 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:59.688 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:59.690 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:59.691 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:03.368 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:03.368 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:08.370 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:08.371 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:08.372 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:11.784 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:11.784 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:16.786 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:16.787 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:16.787 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:20.050 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:20.051 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:25.052 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:25.054 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:25.055 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:28.563 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:28.564 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:33.566 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:33.567 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:33.568 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:36.760 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:36.761 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:41.762 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:41.763 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:41.764 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:44.855 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:44.856 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:49.857 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:49.858 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:49.859 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:53.490 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:53.491 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:58.492 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:58.494 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:58.494 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:01.951 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:01.952 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:06.953 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:06.954 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:06.955 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:10.156 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:10.157 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:15.158 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:15.160 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:15.160 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:18.489 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:18.490 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:23.494 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:23.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:23.496 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:26.815 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:26.815 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:31.817 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:31.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:31.819 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:35.066 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:35.067 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:40.069 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:40.070 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:40.071 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:43.631 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:43.632 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:48.633 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:48.645 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:48.646 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:51.752 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:51.753 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:56.755 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:56.756 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:56.757 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:00.371 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:00.372 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:05.374 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:05.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:05.376 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:09.332 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:09.333 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:14.335 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:14.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:14.337 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:17.657 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:17.657 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:22.659 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:22.660 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:22.661 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:26.028 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:26.029 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:31.030 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:31.031 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:31.031 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:34.389 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:34.391 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:39.392 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:39.393 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:39.394 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:42.867 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:42.868 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:47.871 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:47.872 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:47.873 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:51.319 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:51.320 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:56.330 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:56.331 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:56.332 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:59.482 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:59.483 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:35:04.484 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:04.486 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:04.487 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:07.879 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:35:07.880 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:35:12.881 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:12.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:12.883 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:16.204 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:35:16.205 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:35:21.207 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_sync_disable ... FAIL 2020-05-24T15:35:21.208 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-59 2020-05-24T15:35:22.069 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-60 2020-05-24T15:35:24.157 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-61 2020-05-24T15:35:38.354 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:38.355 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:38.356 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:44.032 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.18.5"}]} 2020-05-24T15:35:44.033 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-59 2020-05-24T15:35:44.034 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:44.034 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:44.035 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:44.939 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:46:55.158 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:44.952 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:46:55.172 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:45.261 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:46:55.481 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:47.843 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.18.5"}}}] 2020-05-24T15:35:47.844 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-59 2020-05-24T15:35:47.844 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:47.845 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:47.846 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:50.928 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.18.5"}]} 2020-05-24T15:35:50.928 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-59 2020-05-24T15:35:50.929 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:50.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:50.930 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:53.604 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.18.5"}}}] 2020-05-24T15:35:53.604 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-59 2020-05-24T15:35:53.605 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:53.605 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:53.606 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:56.143 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.18.5"}]} 2020-05-24T15:35:56.144 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-59 2020-05-24T15:35:56.145 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:56.145 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:56.146 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:00.296 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.18.5"}}}] 2020-05-24T15:36:00.297 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-59 2020-05-24T15:36:00.298 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:00.299 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:00.300 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:03.373 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.25.5"}]} 2020-05-24T15:36:03.374 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-59 2020-05-24T15:36:03.374 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:03.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:03.376 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:06.394 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.25.5"}}}] 2020-05-24T15:36:06.395 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-59 2020-05-24T15:36:06.396 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:06.410 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:06.411 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:08.862 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.25.5"}]} 2020-05-24T15:36:08.863 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-59 2020-05-24T15:36:08.864 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:08.864 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:08.865 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.25.5"}}}] 2020-05-24T15:36:11.763 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-59 2020-05-24T15:36:11.764 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:11.765 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:11.765 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:14.537 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.25.5"}]} 2020-05-24T15:36:14.537 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-59 2020-05-24T15:36:14.538 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:14.538 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:14.539 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:17.130 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:46:20.314 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:36:17.134 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:46:20.318 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:36:17.402 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:46:20.585 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:36:17.875 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:17.876 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:36:22.877 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:22.878 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:22.879 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:26.861 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:26.862 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:36:31.863 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:31.864 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:31.865 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:35.365 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:35.366 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:36:40.382 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:40.383 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:40.384 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:44.311 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:44.312 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:36:49.315 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:49.316 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:49.317 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:52.749 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:52.750 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:36:57.759 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:57.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:57.762 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:01.111 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:01.112 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:37:06.114 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:06.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:06.116 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:09.774 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:09.774 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:37:14.777 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:14.778 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:14.779 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:18.329 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:18.330 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:37:23.332 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:23.333 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:23.334 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:26.856 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:26.857 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:37:31.858 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:31.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:31.860 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:35.632 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:35.633 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:37:39.356 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:47:42.540 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:39.358 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:47:42.542 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:39.463 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:47:42.646 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:40.635 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:40.636 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:40.637 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:44.352 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:44.353 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:37:49.359 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:49.360 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:49.360 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:52.708 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:52.709 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:37:57.372 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:49:07.585 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:57.372 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:49:07.587 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:57.451 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:49:07.670 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:57.711 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:57.712 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:57.713 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:01.336 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:01.336 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:38:06.338 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:06.339 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:06.340 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:10.034 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:10.035 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:38:15.036 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:15.037 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:15.038 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:18.302 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:18.303 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:38:23.304 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:23.306 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:23.306 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:26.817 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:26.818 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:38:31.819 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:31.820 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:31.821 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:35.370 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:35.371 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:38:40.374 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:40.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:40.376 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:42.739 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:49:52.959 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:42.784 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:49:53.004 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:42.949 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:49:53.168 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:44.139 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:44.141 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:38:49.142 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:49.143 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:49.144 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:52.551 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:52.552 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:38:57.554 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:57.555 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:57.556 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:00.839 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:00.839 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:39:05.841 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:05.843 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:05.844 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:09.243 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:09.243 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:39:09.605 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:49:12.789 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:39:09.607 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:49:12.791 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:39:09.665 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:49:12.849 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:39:14.245 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:14.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:14.247 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:17.455 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:17.456 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:39:22.462 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:22.463 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:22.464 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:25.757 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:25.758 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:39:30.759 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:30.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:30.761 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:34.756 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:34.757 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:39:39.769 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:39.771 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:39.772 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:43.542 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:43.543 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:39:48.545 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:48.546 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:48.547 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:52.072 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:52.072 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:39:57.073 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:57.074 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:57.075 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:00.498 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:00.499 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:40:05.500 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:05.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:05.502 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:09.472 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:09.473 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:40:14.486 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:14.487 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:14.488 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:18.205 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:18.206 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:40:23.207 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:23.208 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:23.209 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:26.850 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:26.851 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:40:31.853 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:31.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:31.859 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:35.712 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:35.713 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:40:40.715 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:40.716 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:40.718 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:44.786 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:44.786 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:40:49.788 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:49.789 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:49.790 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:53.500 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:53.501 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:40:58.503 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:58.504 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:58.520 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:01.876 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:01.877 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:41:06.878 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:06.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:06.880 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:10.296 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:10.297 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:41:15.298 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:15.299 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:15.300 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:18.787 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:18.788 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:41:23.789 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:23.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:23.797 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:27.416 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:27.417 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:41:32.420 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:32.422 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:32.423 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:36.051 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:36.052 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:41:41.066 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:41.067 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:41.067 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:44.923 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:44.924 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:41:49.925 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:49.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:49.927 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:53.177 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:53.178 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:41:58.184 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:58.186 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:58.187 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:01.970 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:01.971 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:42:06.973 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:06.975 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:06.976 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:10.385 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:10.386 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:42:15.388 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:15.389 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:15.391 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:19.040 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:19.040 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:42:24.041 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:24.042 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:24.043 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:27.506 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:27.507 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:42:32.508 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:32.509 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:32.510 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:36.412 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:36.413 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:42:39.288 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:53:49.508 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:39.300 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:53:49.520 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:39.621 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:53:49.839 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:41.414 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:41.415 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:41.416 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:44.749 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:42:49.751 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:49.752 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:49.753 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:53.633 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:53.634 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:42:58.635 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:58.636 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:58.637 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:02.386 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:02.387 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:43:07.388 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:07.389 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:07.389 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:10.607 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:53:13.791 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:43:10.613 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:53:13.798 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:43:10.802 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:53:13.986 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:43:10.810 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:10.811 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:43:15.812 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:15.813 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:15.814 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:19.451 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:19.452 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:43:24.453 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:24.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:24.454 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:27.929 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:27.930 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:43:32.938 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:32.939 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:32.940 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:36.619 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:36.620 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:43:41.622 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:41.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:41.623 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:45.194 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:45.194 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:43:50.196 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:50.197 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:50.198 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:54.070 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:54.070 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:43:59.072 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:59.074 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:59.076 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:03.040 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:03.041 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:44:08.042 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:08.044 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:08.044 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:11.880 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:11.882 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:44:16.883 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:16.884 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:16.885 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:20.461 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:20.461 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:44:25.466 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:25.467 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:25.467 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:29.263 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:29.263 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:44:34.265 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:34.266 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:34.267 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:37.927 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:37.928 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:44:42.931 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:42.932 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:42.933 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:46.111 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:46.111 WARNING:rgw_multi.tests:bucket fwzbiz-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.25.5 target= 2020-05-24T15:44:51.113 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_sync_enable_right_after_disable ... FAIL 2020-05-24T15:44:51.115 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-62 2020-05-24T15:44:52.112 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-63 2020-05-24T15:44:53.560 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-64 2020-05-24T15:45:06.368 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:06.369 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:06.370 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=9420dac7-f825-4d7d-a354-f516953dfe80 2020-05-24T15:45:13.639 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590335094.630970_235.1","last_update":"2020-05-24 15:44:54.630970Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:45:13.640 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:45:13.641 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:13.641 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:13.647 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:16.307 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590335094.630970_235.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:44:54.630970Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:45:16.308 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:45:16.308 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:45:16.309 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:16.311 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:16.312 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:20.741 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590335094.630970_235.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:44:54.630970Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:45:20.742 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:45:20.743 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:45:20.746 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:20.747 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:20.748 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:24.119 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590335094.630970_235.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:44:54.630970Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:45:24.120 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:45:24.120 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:24.121 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:24.121 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:28.136 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.16.5"}]} 2020-05-24T15:45:28.137 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-62 2020-05-24T15:45:28.138 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:28.139 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:28.139 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:30.942 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.16.5"}}}] 2020-05-24T15:45:30.942 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-62 2020-05-24T15:45:30.943 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:30.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:30.944 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:34.559 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.16.5"}]} 2020-05-24T15:45:34.560 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-62 2020-05-24T15:45:34.560 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:34.561 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:34.562 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:38.484 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.16.5"}}}] 2020-05-24T15:45:38.486 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-62 2020-05-24T15:45:38.487 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:38.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:38.489 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:41.507 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000008.16.5"}]} 2020-05-24T15:45:41.508 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-62 2020-05-24T15:45:41.509 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:41.509 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:41.510 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:45.530 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.16.5"}}}] 2020-05-24T15:45:45.530 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-62 2020-05-24T15:45:45.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:45.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:45.532 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:48.921 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.19.5"}]} 2020-05-24T15:45:48.922 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-62 2020-05-24T15:45:48.922 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:48.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:48.924 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:51.788 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.19.5"}}}] 2020-05-24T15:45:51.789 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-62 2020-05-24T15:45:51.790 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:51.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:51.792 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:55.183 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.19.5"}]} 2020-05-24T15:45:55.184 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-62 2020-05-24T15:45:55.184 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:55.185 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:55.185 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:57.912 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.19.5"}}}] 2020-05-24T15:45:57.913 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-62 2020-05-24T15:45:57.913 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:57.914 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:57.914 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:00.289 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000008.19.5"}]} 2020-05-24T15:46:00.290 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-62 2020-05-24T15:46:00.290 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:00.291 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:00.291 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:03.581 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:03.582 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:46:08.584 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:08.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:08.585 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:12.260 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:12.261 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:46:17.263 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:17.264 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:17.265 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:18.742 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:57:28.961 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:46:18.743 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:57:28.963 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:46:18.879 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:01:57:29.094 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:46:20.899 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:20.899 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:46:25.901 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:25.902 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:25.903 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:29.709 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:29.710 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:46:34.710 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:34.711 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:34.712 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:38.027 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:38.027 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:46:43.031 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:43.032 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:43.032 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:46.597 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:46.598 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:46:51.600 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:51.601 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:51.601 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:55.308 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:55.309 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:47:00.311 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:00.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:00.314 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:03.890 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:03.891 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:47:08.894 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:08.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:08.896 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:12.478 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:12.479 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:47:17.483 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:17.484 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:17.485 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:20.736 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:20.736 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:47:25.738 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:25.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:25.739 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:29.618 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:29.619 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:47:30.424 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:57:33.607 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:47:30.428 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:57:33.612 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:47:30.689 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:01:57:33.873 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:47:34.620 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:34.622 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:34.623 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:38.226 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:38.227 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:47:43.229 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:43.230 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:43.232 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:46.923 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:46.924 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:47:51.925 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:51.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:51.927 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:55.577 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:55.578 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:48:00.580 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:00.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:00.581 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:04.571 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:04.572 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:48:09.573 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:09.574 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:09.575 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:13.032 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:13.033 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:48:18.034 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:18.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:18.037 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:21.502 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:21.503 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:48:26.505 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:26.506 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:26.507 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:30.597 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:30.598 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:48:35.599 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:35.600 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:35.601 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:39.106 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:39.107 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:48:44.110 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:44.111 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:44.111 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:47.514 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:47.515 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:48:52.516 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:52.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:52.518 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:56.241 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:56.242 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:49:01.244 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:01.245 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:01.246 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:04.941 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:04.942 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:49:09.945 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:09.946 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:09.947 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:13.486 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:13.487 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:49:18.489 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:18.490 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:18.491 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:22.441 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:22.442 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:49:27.444 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:27.446 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:27.446 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:31.134 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:31.135 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:49:31.840 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:00:42.059 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:31.851 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:00:42.069 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:32.046 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:00:42.265 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:36.147 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:36.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:36.149 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:39.726 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:39.727 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:49:44.730 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:44.731 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:44.732 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:48.611 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:48.611 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:49:53.613 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:53.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:53.615 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:57.021 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:57.022 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:50:02.024 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:02.026 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:02.027 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:05.654 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:00:08.833 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:50:05.657 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:00:08.840 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:50:05.785 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:05.786 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:50:05.798 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:00:08.983 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:50:10.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:10.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:10.789 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:14.688 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:14.689 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:50:19.690 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:19.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:19.692 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:23.796 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:23.797 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:50:28.800 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:28.801 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:28.802 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:32.706 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:32.707 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:50:37.711 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:37.712 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:37.713 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:41.398 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:41.399 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:50:46.401 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:46.402 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:46.402 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:50.148 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:50.149 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:50:55.150 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:55.151 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:55.152 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:58.412 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:58.413 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:51:03.418 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:03.419 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:03.419 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:07.046 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:07.047 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:51:12.048 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:12.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:12.051 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:15.772 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:15.772 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:51:20.774 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:20.775 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:20.776 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:24.715 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:24.716 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:51:29.718 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:29.719 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:29.719 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:33.371 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:33.372 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:51:38.374 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:38.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:38.375 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:41.777 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:41.778 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:51:46.779 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:46.780 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:46.781 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:50.660 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:50.661 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:51:55.662 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:55.663 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:55.664 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:59.124 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:59.125 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:52:04.126 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:04.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:04.128 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:07.985 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:07.986 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:52:12.987 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:12.988 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:12.989 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:16.434 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:16.435 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:52:21.437 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:21.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:21.439 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:25.675 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:25.676 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:52:30.677 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:30.679 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:30.679 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:34.527 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:34.528 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:52:39.530 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:39.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:39.531 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:43.163 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:43.164 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:52:48.166 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:48.167 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:48.168 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:51.856 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:51.857 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:52:56.860 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:56.861 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:56.862 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:00.357 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:00.358 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:53:05.359 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:05.362 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:05.363 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:09.360 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:09.361 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:53:14.370 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:14.372 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:14.372 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:17.691 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:17.692 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:53:22.693 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:22.694 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:22.695 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:26.505 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:26.508 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:53:31.511 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:31.512 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:31.513 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:35.155 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:35.155 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:53:40.157 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:40.158 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:40.159 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:43.996 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:43.997 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:53:48.998 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:48.999 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:48.999 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:52.471 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:52.472 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:53:57.473 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:57.474 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:57.475 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:01.329 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:54:01.331 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:54:06.333 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:06.334 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:06.335 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:09.910 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:54:09.910 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:54:14.912 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:14.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:14.914 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:18.850 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:54:18.851 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:54:23.852 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:23.853 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:23.854 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:27.510 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:54:27.511 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:54:32.513 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:32.514 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:32.521 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:36.322 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:54:36.323 WARNING:rgw_multi.tests:bucket fwzbiz-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.19.5 target= 2020-05-24T15:54:39.604 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:05:49.824 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:54:39.614 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:05:49.833 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:54:39.842 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:05:50.055 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:54:41.326 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_sync_disable_enable ... FAIL 2020-05-24T15:54:41.328 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-65 2020-05-24T15:54:42.179 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=fwzbiz-66 2020-05-24T15:54:43.979 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=fwzbiz-67 2020-05-24T15:55:04.162 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:04.163 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:04.164 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:09.262 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000018.15.5"}]} 2020-05-24T15:55:09.263 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-65 2020-05-24T15:55:09.264 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:09.264 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:09.265 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:12.335 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000018.15.5"}}}] 2020-05-24T15:55:12.335 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-65 2020-05-24T15:55:12.347 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:12.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:12.349 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:16.087 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000018.15.5"}]} 2020-05-24T15:55:16.088 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-65 2020-05-24T15:55:16.088 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:16.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:16.090 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:20.141 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000018.15.5"}}}] 2020-05-24T15:55:20.142 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-65 2020-05-24T15:55:20.143 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:20.144 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:20.144 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:23.807 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000018.15.5"}]} 2020-05-24T15:55:23.808 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-65 2020-05-24T15:55:23.808 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:23.809 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:23.810 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:27.581 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000018.15.5"}}}] 2020-05-24T15:55:27.581 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-65 2020-05-24T15:55:27.582 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:27.582 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:27.583 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:30.401 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T15:55:30.401 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-65 2020-05-24T15:55:30.407 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:30.408 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:30.408 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:34.378 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"MULTIPART","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.3.5"}}}] 2020-05-24T15:55:34.379 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-65 2020-05-24T15:55:34.379 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:34.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:34.380 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:38.042 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T15:55:38.043 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-65 2020-05-24T15:55:38.043 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:38.044 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:38.044 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:40.811 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"MULTIPART","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.3.5"}}}] 2020-05-24T15:55:40.812 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-65 2020-05-24T15:55:40.812 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:40.813 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:40.814 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:44.752 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T15:55:44.752 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-65 2020-05-24T15:55:44.753 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:44.754 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:44.754 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:48.919 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:55:48.920 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:55:51.128 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:05:54.312 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:51.157 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:05:54.341 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:51.472 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:05:54.643 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:53.924 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:53.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:53.928 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:57.743 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:55:57.744 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:02.745 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:02.746 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:02.747 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:06.533 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:06.533 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:11.535 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:11.537 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:11.538 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:12.116 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:07:22.335 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:12.126 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:07:22.346 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:12.339 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:07:22.557 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:15.152 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:15.153 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:20.155 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:20.156 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:20.157 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:23.479 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:23.479 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:28.481 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:28.482 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:28.483 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:32.164 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:32.165 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:37.167 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:37.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:37.169 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:41.013 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:41.014 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:46.015 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:46.016 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:46.017 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:49.963 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:49.964 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:54.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:54.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:54.966 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:58.906 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:58.907 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:56:59.583 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:07:02.767 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:59.596 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:07:02.781 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:59.723 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:07:02.908 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T15:57:03.909 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:03.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:03.911 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:07.511 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:07.512 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:57:12.514 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:12.515 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:12.516 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:16.002 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:16.003 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:57:21.005 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:21.006 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:21.007 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:24.743 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:24.744 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:57:29.749 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:29.750 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:29.751 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:33.622 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:33.623 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:57:38.625 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:38.626 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:38.627 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:42.538 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:42.539 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:57:47.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:47.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:47.549 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:51.311 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:51.312 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:57:56.313 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:56.314 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:56.315 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:59.986 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:59.987 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:58:04.989 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:04.989 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:04.990 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:08.711 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:08.712 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:58:13.714 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:13.715 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:13.716 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:17.463 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:17.464 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:58:22.466 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:22.467 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:22.468 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:26.114 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:26.115 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:58:31.117 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:31.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:31.119 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:34.828 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:34.829 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:58:39.834 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:39.835 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:39.835 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:43.701 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:43.702 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:58:48.703 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:48.704 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:48.705 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:52.516 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:52.517 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:58:57.519 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:57.520 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:57.521 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:01.418 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:01.419 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:59:06.420 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:06.421 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:06.422 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:10.127 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:10.128 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:59:15.129 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:15.130 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:15.131 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:18.806 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:18.808 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:59:23.809 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:23.810 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:23.811 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:27.322 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:27.322 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:59:32.324 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:32.325 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:32.326 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:36.218 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:36.219 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:59:41.222 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:41.223 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:41.224 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:45.279 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:45.280 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:59:50.282 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:50.283 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:50.283 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:53.833 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:53.834 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T15:59:58.835 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:58.837 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:58.837 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:02.514 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:02.515 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:00:07.516 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:07.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:07.527 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:10.999 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:11.000 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:00:16.005 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:16.006 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:16.007 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:19.784 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:19.785 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:00:24.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:24.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:24.789 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:28.689 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:28.690 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:00:33.692 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:33.693 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:33.694 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:37.642 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:37.642 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:00:42.644 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:42.646 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:42.646 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:46.416 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:46.417 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:00:51.419 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:51.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:51.422 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:55.732 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:55.733 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:01:00.735 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:00.736 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:00.736 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:04.378 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:04.378 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:01:09.379 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:09.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:09.381 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:13.166 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:13.167 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:01:18.169 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:18.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:18.171 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:21.689 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:21.690 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:01:26.691 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:26.692 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:26.693 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:30.888 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:30.889 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:01:35.899 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:35.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:35.901 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:39.762 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:39.763 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:01:44.765 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:44.766 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:44.767 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:48.584 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:48.584 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:01:53.603 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:53.604 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:53.605 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:57.225 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:57.225 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:02:02.228 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:02.229 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:02.229 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:05.988 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:05.989 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:02:10.990 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:10.991 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:10.992 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:14.769 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:14.769 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:02:19.771 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:19.772 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:19.772 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:23.692 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:23.692 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:02:28.694 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:28.695 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:28.695 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:31.981 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:31.982 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:02:36.983 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:36.984 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:36.985 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:40.938 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:40.939 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:02:45.947 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:45.948 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:45.949 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:49.490 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:49.491 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:02:54.493 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:54.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:54.501 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:57.802 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:57.803 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:03:00.522 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:14:10.741 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:00.596 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:14:10.815 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:00.836 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:14:11.056 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:02.805 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:02.807 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:02.808 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:06.684 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:06.685 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:03:08.642 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:14:18.861 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:08.656 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:14:18.876 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:08.737 INFO:tasks.ceph.c1.mon.a.smithi019.stderr:==00:02:14:18.957 9344== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:11.686 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:11.687 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:11.688 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:15.692 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:15.693 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:03:20.694 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:20.694 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:20.695 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:24.613 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:24.614 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:03:29.615 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:29.617 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:29.618 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:33.465 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:33.466 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:03:38.468 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:38.469 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:38.470 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:42.102 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:42.103 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:03:47.106 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:47.107 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:47.107 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:51.102 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:51.102 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:03:56.103 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:56.104 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:56.105 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:57.179 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:14:00.363 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:57.182 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:14:00.366 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:57.486 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:14:00.671 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:59.565 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:59.566 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:04:04.567 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:04.569 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:04.570 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:08.235 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:04:08.235 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:04:12.033 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:14:15.209 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T16:04:12.042 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:14:15.226 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T16:04:12.115 INFO:tasks.ceph.c2.mon.a.smithi193.stderr:==00:02:14:15.300 10211== Warning: unimplemented fcntl command: 1036 2020-05-24T16:04:13.237 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:13.238 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:13.239 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:16.801 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:04:16.802 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:04:21.803 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:21.805 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:21.805 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:25.688 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:04:25.689 WARNING:rgw_multi.tests:bucket fwzbiz-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T16:04:30.691 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_multipart_object_sync ... FAIL 2020-05-24T16:04:30.693 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-68 2020-05-24T16:04:32.730 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:32.731 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:32.732 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:35.476 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=9420dac7-f825-4d7d-a354-f516953dfe80 2020-05-24T16:04:37.388 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590336271.548659_255.1","last_update":"2020-05-24 16:04:31.548659Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T16:04:37.389 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T16:04:37.389 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:37.390 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:37.391 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590336271.548659_255.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:31.548659Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:04:39.871 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T16:04:39.872 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T16:04:39.873 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:39.874 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:39.875 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:42.746 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590336271.548659_255.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:31.548659Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:04:42.747 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T16:04:42.748 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T16:04:42.749 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:42.750 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:42.751 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:46.454 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590336271.548659_255.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:31.548659Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:04:46.454 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T16:04:46.455 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-68', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:46.456 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-68', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:46.456 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-68 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:49.591 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000004.7.5"}]} 2020-05-24T16:04:49.592 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-68 2020-05-24T16:04:49.592 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:49.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:49.594 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-68 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:52.561 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"testobj-sse-kms","instance":"","ns":""},"count":2},"inc_marker":{"position":"0#00000000004.7.5"}}}] 2020-05-24T16:04:52.562 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-68 2020-05-24T16:04:53.306 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_encrypted_object_sync ... ok 2020-05-24T16:04:53.307 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=fwzbiz-69 2020-05-24T16:04:57.655 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:57.656 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:57.657 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:No --period given, using current period=9420dac7-f825-4d7d-a354-f516953dfe80 2020-05-24T16:05:02.969 INFO:teuthology.orchestra.run.smithi019.stdout:[{"marker":"1_1590336294.128049_259.1","last_update":"2020-05-24 16:04:54.128049Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T16:05:02.970 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T16:05:02.971 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:02.971 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:02.972 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:05.771 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590336294.128049_259.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:54.128049Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:05:05.772 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T16:05:05.772 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T16:05:05.773 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:05.773 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:05.775 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:09.542 INFO:teuthology.orchestra.run.smithi019.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590336294.128049_259.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:54.128049Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:05:09.543 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T16:05:09.544 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T16:05:09.545 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:09.546 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:09.547 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"9420dac7-f825-4d7d-a354-f516953dfe80","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590336294.128049_259.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:54.128049Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1590328801.559787_129.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2020-05-24 14:00:01.559787Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:05:13.444 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T16:05:13.445 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:13.447 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:13.448 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:17.965 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000009.11.5"}]} 2020-05-24T16:05:17.966 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-69 2020-05-24T16:05:17.967 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:17.967 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:17.968 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:20.506 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"b","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000009.11.5"}}}] 2020-05-24T16:05:20.507 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=fwzbiz-69 2020-05-24T16:05:20.507 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:20.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:20.508 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:24.040 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000009.11.5"}]} 2020-05-24T16:05:24.041 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-69 2020-05-24T16:05:24.042 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:24.042 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:24.043 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:28.254 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"b","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000009.11.5"}}}] 2020-05-24T16:05:28.255 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=fwzbiz-69 2020-05-24T16:05:28.256 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:28.257 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:28.258 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:32.066 INFO:teuthology.orchestra.run.smithi019.stdout:{"markers":[{"key":0,"val":"00000000009.11.5"}]} 2020-05-24T16:05:32.067 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-69 2020-05-24T16:05:32.067 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:32.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:32.069 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:35.990 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"c","instance":"","ns":""},"count":3},"inc_marker":{"position":"00000000009.11.5"}}}] 2020-05-24T16:05:35.991 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=fwzbiz-69 2020-05-24T16:05:35.992 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:35.993 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:35.993 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:39.905 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000009.12.3"}]} 2020-05-24T16:05:39.905 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-69 2020-05-24T16:05:39.906 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:39.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:39.907 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:43.026 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000009.12.3"}}}] 2020-05-24T16:05:43.027 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=fwzbiz-69 2020-05-24T16:05:43.028 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:43.029 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:43.030 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:46.681 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000009.12.3"}]} 2020-05-24T16:05:46.682 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-69 2020-05-24T16:05:46.683 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:46.683 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:46.684 INFO:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:49.456 INFO:teuthology.orchestra.run.smithi019.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"c","instance":"","ns":""},"count":3},"inc_marker":{"position":"00000000009.12.3"}}}] 2020-05-24T16:05:49.457 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=fwzbiz-69 2020-05-24T16:05:49.457 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:49.458 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:49.459 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:53.398 INFO:teuthology.orchestra.run.smithi193.stdout:{"markers":[{"key":0,"val":"00000000009.12.3"}]} 2020-05-24T16:05:53.399 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=fwzbiz-69 2020-05-24T16:05:53.400 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:53.400 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:53.401 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:57.512 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:05:57.512 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:06:02.514 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:02.515 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:02.516 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:06.438 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:06.439 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:06:11.440 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:11.441 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:11.442 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:14.869 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:14.870 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:06:19.872 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:19.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:19.874 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:24.095 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:24.096 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:06:29.100 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:29.101 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:29.101 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:32.982 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:32.983 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:06:37.984 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:37.986 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:37.987 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:42.157 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:42.158 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:06:47.167 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:47.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:47.169 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:50.692 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:50.693 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:06:55.697 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:55.698 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:55.699 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:59.578 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:59.579 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:07:04.582 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:04.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:04.584 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:08.231 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:08.232 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:07:13.234 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:13.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:13.236 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:17.190 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:17.191 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:07:22.193 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:22.194 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:22.195 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:25.896 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:25.897 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:07:30.898 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:30.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:30.900 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:34.944 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:34.945 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:07:39.946 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:39.947 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:39.948 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:43.834 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:43.835 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:07:48.842 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:48.843 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:48.844 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:52.920 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:52.921 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:07:57.926 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:57.927 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:57.928 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:01.659 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:01.660 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:08:06.664 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:06.665 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:06.666 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:10.443 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:10.444 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:08:15.446 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:15.447 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:15.448 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:19.097 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:19.098 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:08:24.100 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:24.101 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:24.102 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:28.110 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:28.111 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:08:33.113 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:33.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:33.115 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:36.871 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:36.872 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:08:41.874 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:41.875 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:41.876 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:46.054 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:46.055 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:08:51.057 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:51.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:51.059 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:54.701 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:54.702 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:08:59.703 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:59.704 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:59.705 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:03.407 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:03.408 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:09:08.413 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:08.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:08.416 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:12.190 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:12.192 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:09:17.193 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:17.195 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:17.196 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:20.804 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:20.805 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:09:25.808 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:25.809 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:25.810 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:29.640 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:29.641 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:09:34.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:34.657 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:34.658 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:38.536 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:38.537 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:09:43.539 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:43.541 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:43.542 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:47.267 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:47.268 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:09:52.272 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:52.274 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'fwzbiz-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:52.275 INFO:teuthology.orchestra.run.smithi193:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket fwzbiz-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:56.037 INFO:teuthology.orchestra.run.smithi193.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:56.038 WARNING:rgw_multi.tests:bucket fwzbiz-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.12.3 target= 2020-05-24T16:09:57.916 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-05-24T16:09:57.917 DEBUG:teuthology.task.console_log:Killing console logger for smithi019 2020-05-24T16:09:57.918 DEBUG:teuthology.task.console_log:Killing console logger for smithi193 2020-05-24T16:09:57.919 DEBUG:teuthology.task.console_log:Killing console logger for smithi019 2020-05-24T16:09:57.919 DEBUG:teuthology.task.console_log:Killing console logger for smithi193 2020-05-24T16:09:57.919 DEBUG:teuthology.exit:Finished running handlers