Fetch all query rows before processing

contrail-logs takes lot of time processing the query result. Profiling
of the contrail-logs script showed that lot of time is being spent on
xmltodict.parse() and this impacts the overall time taken to fetch all
the rows from the anlaytics-api. TTL for the query result is set to 5 minutes
in redis. Therefore, if the number of records is more for the specified time
duration in the query, then it is possible that some rows may expire
before they are read and hence contrail-logs would display only partial
query result.
To avert this issue, contrail-logs should fetch the query result
completely before processing it.

Change-Id: Ic54731836675009895d69a26514d5699da96bf73
Closes-Bug: #1632446
(cherry picked from commit 71b508d)

src/opserver/log.py

@@ -74,7 +74,13 @@ def run(self):
                 result = self.query()
                 if result == -1:
                     return
-                self.display(result)
+                # Accumulate the result before processing it as the
+                # formatting of result can be cpu intensive and hence would
+                # affect the overall time taken to fetch the result from the
+                # analytics-api. Since the query result ttl is set to 5 min
+                # in redis, it is necessary to improve the read throughput.
+                result_list = self.read_result(result)
+                self.display(result_list)
         except KeyboardInterrupt:
             return
 
@@ -496,6 +502,15 @@ def output(self, log_str, sandesh_level):
             print log_str
     #end output
 
+    def read_result(self, result_gen):
+        if not result_gen:
+            return
+        result_list = []
+        for r in result_gen:
+            result_list.append(r)
+        return result_list
+    # end read_result
+
     def display(self, result):
         if result == [] or result is None:
             return