Geek asks and answers

Real-time analytics with BigQuery

Is there a way to run real-time analytics using BigQuery? I used the CSV download option, which launches the job and downloads offline data that can be analyzed after the download is complete. But the BigQuery post mentions the use of BigQuery for real-time analytics. How can this be achieved? Can we add (without updates) data from the Google Cloud database to BigQuery in trickle mode for real-time analytics?

As a side element, I noticed that loading BigQuery CSV data is about an order of magnitude slower than LucidDB and InfiniDB working on my local PC using a 10 GB data file. At the end of the assignment, BigQuery took 34 minutes versus 5 minutes on InfiniDB and LucidDB. Query execution time (for simple aggregates) in BigQuery is twice as slow compared to InfiniDB (6 seconds versus 3 seconds versus a 10 GB file loaded with approximately 30 million records), but better than LucidDB.

+3
source share
5 answers
+2

. . .

, . , - , , .

+1

BiqQuery - python, Google. .

0

SQL- , API Stride, SQL- , . Stride SQL, PipelineDB, fork PostgreSQL PostgreSQL .

The good thing about continuous SQL queries in threads for your level of real-time analytics is that if you have a real-time need, then by definition you already know the queries you want to run, so continuous queries speed up and greatly simplify your real time while reducing costs associated with storing extraneous granular data.

0
source

More articles:


All Articles