I have been writing in the form of series quite a bit. Recently, I wrote a series of posts for SQL Azure DB which really helped the readers to understand the subject. Even, I had to write the same series for SQL on Azure VMs but somehow I couldn’t finish that. Hopefully, I will finish that in the near future.
For now, lets talk on what’s NoSQL all about. There has been lots of discussion and publicity around this subject over the past few years and it’s gaining lot of popularity because of it’s efficiency. If you are in the world where SQL/Oracle/DB2 etc. are the only resorts for the data storage, then you really need to upgrade. It doesn’t mean RDBMS systems like SQL/Oracle/DB2 etc. are going out of trend , it just means that now for different needs we need to pick different database systems. People no longer rely just on RDBMS systems.
I have been reading some of the really amazing blog posts,written by David Campbell. For now, I’m going to diverge a little bit from the subject. Using the technique mentioned by David , I will explain the bigger picture.The best way to understand where the data world is going, is to understand the below mentioned data categories:
1. Operational Data
2. Analytical Data
3. Streaming Data
Operational Data – Operational data is the data used by the applications to maintain their state e.g. Payment data/Customer information like we have on OLTP or non-transaction systems. However, slowly people realized the real value of historical data which could be used to understand the trends for higher customer satisfaction or for building the business strategies. That’s how all the technologies for data warehousing started gaining traction.
Analytical Data – This is the read only data , analyzed using Data warehouse or Big Data systems to understand the business trends and historical data. Due to the huge volume of the data , Big Data is gaining the trends as it needs extreme hardware capacity to analyze PBs of data.However, for the smaller systems traditional OLAP systems work perfectly fine.
Streaming Data – In this modern world, people want analytics in the real time e.g. fitness tracker on the people’s wrists, toll payment devices in the cars or sensors on the oil well etc. One way is to store the entire data and then do the analysis but sometimes delay in the processing is not affordable e.g. if oil company wants to raise an alert if the pressure in the well is increasing or if you wanted to know how many cars passed through a specific city in last 30 minutes. There has to be a provision to read live stream and make sense out of that. This is the world of IOT.
Understanding the above terms , was really important for everyone working in the data platform. There is a plethora of companies working towards making data platform really a happening world. Have you heard about 3Vs of data – Volume , Variety , Velocity? In today’s world it’s difficult to manage these 3 Vs in RDBMS. When it comes to RDBMS, we talk about structured data in the form of Tables/Columns. Anything and everything you want to store in the RDBMS systems has to be stored in the form of Tables/Columns. How about the data like flat files/JSON/ telemetry data where there is no structure? As I shared in the beginning, for different types of data we need different database systems.
In a nutshell, there are five major categories of database technologies:
1. Relational databases (RDBMS)
2. Document Store
3. Column family Store
4. Key Value Store
5. Graph Databases
We will discuss each of the category for the NoSQL in detail, in upcoming posts.
Disclaimer: The views expressed on this website/blog are mine alone and do not reflect the views of my company. All postings on this blog are provided “AS IS” with no warranties, and confers no rights.