A large amount of data? The answer is BigQuery

Google_bigqueryGoogle is giant, and the search engine is not its main product. It is incorporated and Cloud Computing, which means that you can rent servers or a fleet of servers. BigQuery is a service integrated in the Google Cloud platform, that we can use to run huge data or analysis on Big Data.

What is BigQuery?

The analyses are conducted in the SQL language, familiar to us, used on traditional basics. BigQuery focuses primarily on very large data that can not be performed conveniently “in house”. At Reea we use it to store unstructured data, more precisely we capture the data produced by the users on mobile or web applications and further analyze them with BigQuery.

The platform takes advantage of the processing power of Google infrastructure because it provides a high speed. The data must be available in real-time, that is what our leaders and analysts want. I want to analyze, summarize, find something I can develop, to enable informed decision-making for better business results. BigQuery offers this possibility.

Developers always had a problem with converting the large volumes into decisions. Now there are several types of information for the background analysis, such as email or mobile sources.

The costs are low. I have to mention that a terabyte of data is free. If the user is looking for something specific, Google BigQuery can run analyzes up to petabytes. 5 terabytes cost 12 dollars, so companies can invest money to scale up to a petabyte, which appears very high to us. However, the data generated by companies and solutions soar over time and the need increases too.

Concrete examples of use of the platform BigQuery

1. If we process the data derived from an email campaign, we can find out the exact time when someone has opened and email or clicked an item. So if that person checks his emails at 9, I can send one at 8:50, to be sure that this will be the first item in his inbox. The analysis can show us if a subject was improved or the layouts are more efficient than the previous version of them. Therefore, we can achieve better business results.

2. We can export and access session data from Google Analytics Premium account in BigQuery, then using an SQL-like syntax to interrogate all of the data from Google Analytics. Examples include the average number of user interactions before purchasing a product or the purchased products by customers who initially bought product A.

Directions of the companies

Globally, each company chooses the directions in which they develop their solutions. Some of them develop their own solutions on their infrastructure, others use open source frameworks. For example, Spark or Hadoop are some of those which are the most used to manage huge volumes of data. Major customers, utility companies or telecommunications, have the necessary power to run the data “inside”.  They are using these frameworks to make informed Big Data decisions. Other companies use a tool like Google BigQuery Developers, which allows them to run super fast query large data sets.

Reea at Dev Talks Cluj, this year

At the Dev Talks conference of 13th May 2015 in Cluj, I presented a Big Data solution for the analysis of events in real-time. A high volume data storage is an issue that can not be performed with own equipment, but only with high costs. Too many problems assume a massive data analysis. In my presentation, I have focused on the ease of the BigQuery technology, more precisely, how one can go home and run all this information and analysis in SQL.

What are we preparing for the future?

Clipboard01_800

As a first step, after the conference in Cluj, we organised a meetup with free entry in Tîrgu-Mureș. There were over 50 participants very interested in the subject, including students. We would like the students to develop Big Data solutions. We encourage teachers to guide students in choosing topics from the field of modern technology for their graduation thesis. Google BigQuery is free up to one terabyte.

Secondly, we want to lay the foundations of a community that can be improved with the help of Big Data solutions. Public transport can be improved with Big Data and even the information gathered from supermarket shelves. There is also a large Waze community in Romania. We want to improve the system and to create external applications that increase even more the level of comfort for road haulers.

It’s important to keep up with technologies offered by giants from abroad. These technologies can boost the local level. Each developer specializes constantly and each day they learn more and more.

Write a Reply or Comment

Your email address will not be published. Required fields are marked *

my

*


Please do not write personal data in the comments section.