Data Persistence

 Role of data in information systems

Persistence is the continuance of an effect after its cause is removed. In the context of storing data in a computer system, this means that the data survives after the process with which it was created has ended. In other words, for a data store to be considered persistent, it must write to nonvolatile storage. .If you need persistence in your data store, then you need to also understand the four main design approaches that a data store can take and how (or if) these designs provide persistence:


1. Pure in memory , no persistence at all such as memcached or scalaris
2. In memory with periodic snapshots such as Oracle Coherence or Redis
3. Disk based with update in placewrites such as MySQL ISAM or MongoDB

Special Terms

Data is a set of values of subjects with respect to qualitative or quantitative variables. Data and information or knowledge are often used interchangeably; however data becomes information when it is viewed in context or in postanalysis.

Data is distinct pieces of information, usually formatted in a special way. All software is divided into two general categories: data and programs. Programs are collection of instructionsfor manipulating data. 

Data can exist in a variety of forms — as numbers or text on pieces of paper, as bits and bytes stored in electronic memory, or as facts stored in a person's mind. Since the mid-1900s, people have used the word data to mean computer information that is transmitted or stored.

Database Management System

A database management system (DBMS) is a software package designed to define, manipulate, retrieve and manage data in a database. A DBMS generally manipulates the data itself, the data format, field names, record structure and file structure. It also defines rules to validate and manipulate this data. A DBMS relieves users of framing programs for data maintenance. Fourth-generation query languages, such as SQL, are used along with the DBMS package to interact with a database.

A database management system receives instruction from a database administrator (DBA) and accordingly instructs the system to make the necessary changes. These commands can be to load, retrieve or modify existing data from the system.

 Files And Databases

1.Performance can be better than when we do it in a database - To justify this, if you store large files in DB, then it may slow down the performance because a simple query to retrieve the list of files or filename will also load the file data if you used select * in your query. In a files ystem, accessing a file is quite simple and light weight. 

2. Saving the files and downloading them in the file system is much simpler - than it is in a database since a simple "Save As" function will help you out. Downloading can be done by addressing a URL with the location of the saved file. 

3. Migrating data is an easy process - You can just copy and paste the folder to your desired destination while ensuring that write

 

1. ACID consistency - which includes a rollback of an update that is complicated when files are stored outside the database. 

2. Files will be in sync with the database - and cannot be orphaned, which gives you the upper hand in tracking transactions. 

3. Backups automatically include file libraries.

4. More Secure 

Different Types of Databases 

In a hierarchical database management systems (hierarchical DBMSs) model, data is stored in a parent-children relationship nodes. In a hierarchical database, besides actual data, records also contain information about their groups of parent/child relationships. 

 

In a hierarchical database model, data is organized into a tree like structure. The data is stored in form of collection of fields where each field contains onlyone value. The records are linked to each other via links into a parentchildren relationship. In a hierarchical database model, each child record has only one parent. A parent can have multiple children.

 

The hierarchical database system structure was developed by IBM in early 1960s. While hierarchical structure is simple, it is inflexible due to the parent-child one-to-many relationship. Hierarchical databases are widely used to build high performance and availability applications usually in banking and telecommunications industries.

 

Big Data Vs Data Warehouse

 1.Data Warehouse is an architecture of data storing or data repository. Whereas Big Data is a technology to handle huge data and prepare the repository. 

2. Any kind of DBMS data accepted by Data warehouse, whereas Big Data accept all kind of data including transnational data, social media data, machinery data or any DBMS data. 

3. Data warehouse only handles structure data (relational or not relational), but big data can handle structure, non-structure, semi-structured data. 

4. Big data normally used a distributed file system to load huge data in a distributed way, but data warehouse doesn’t have that kind of concept. 

5. From a business point of view, as big data has a lot of data, analytics on that will be very fruitful, and the result will be more meaningful which help to take proper decision for that organization. Whereas Data warehouse mainly helps to analytic on informed information. 

6. Data warehouse means the relational database, so storing, fetching data will be similar with normal SQL query. And big data is not following proper database structure, we need to use hive or spark SQL to see the data by using hive specific query. 

 

SQL Statements, Prepared Statements and Callable Statements

 

Most of the actions you need to perform on a database are done with SQL statements.  

SQL keywords are NOT case sensitive: select is the same as SELECT. 

Some database systems require a semicolon at the end of each SQL statement. 

Semicolon is the standard way to separate each SQL statement in database systems that allow more than one SQL statement to be executed in the same call to the server. 

Here some important SQL statements

 

SELECT - extracts data from a database 

UPDATE - updates data in a database 

DELETE - deletes data from a database 

INSERT INTO - inserts new data into a database 

CREATE DATABASE - creates a new database 

ALTER DATABASE - modifies a database 

CREATE TABLE - creates a new table 

ALTER TABLE - modifies a table 

DROP TABLE - deletes a table 

CREATE INDEX - creates an index (search key) 

DROP INDEX - deletes an index

Information Retrieval

 

Information Retrieval is the activity of obtaining information system resources relevant to an information need from a collection. Searches can be based on full-text or other content-based indexing. Information retrieval is the science of searching for information in a document, searching for documents themselves, and also searching for metadata that describe data, and for databases of texts, images or sounds.

Information retrieval and information filtering are different functions. Information retrieval is intended to support people who are actively seeking or searching for information, as in Internet searching. Information retrieval typically assumes a static or relatively static database against which people search. Search engine companies construct these databases by sending out “spiders” and then indexing the Web pages they find. By contrast, information filtering supports people in the passive monitoring for desired information. It is typically understood to be concerned with an active incoming stream of information objects.