Big Data Analytics

Big data analytics is a process used to extract meaningful insights, such as hidden patterns, correlations and other information from huge datasets. It uses advanced analytics software against large and diverse datasets.

Big data analytics tools employ a variety of approaches including:

• data mining, which sifts through data sets in search of patterns and relationships;
• predictive analytics, which builds models to forecast future developments;
• machine learning, which generates algorithms to analyze large data sets; and 
• deep learning, a more advanced offshoot of machine learning.
• geospatial analytics, which deals with vector, raster and point cloud datasets which can be very large (petabytes or terabytes).

These tools can be provided as a service from major companies such as Amazon, Microsoft, Google and IBM. Other options are to use tools such as the freeware version of RStudio or python libraries developed by the programming community and freely available, such as pandas and scikit-learn. Much GIS software such as ArcGIS is proprietary but there are many open-source packages such as QGIS, SpatialHadoop, PostGIS and GeoServer.

To provide access to this data analytics, Application Programming Interfaces (APIs) can be employed. APIs allow users to interact directly with tools such as machine learning algorithms that are hosted on the internet. These APIs can also be developed using free python libraries like flask and tested using web clients such as Postman.