Lab4 2HiveMongDB Sagar - Cleveland State...

11
MONGODB MONGODB MONGODB MONGODB to install the MongoDB first need to import the public key used by the package management system sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6 Create a list files for the MongoDB Create a list files for the MongoDB Create a list files for the MongoDB Create a list files for the MongoDB echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list Install the MongoD Install the MongoD Install the MongoD Install the MongoDB Packages B Packages B Packages B Packages sudo apt-get install -y mongodb

Transcript of Lab4 2HiveMongDB Sagar - Cleveland State...

Page 1: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

MONGODBMONGODBMONGODBMONGODB

to install the MongoDB first need to import the public key used by the package management system

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv

0C49F3730359A14518585931BC711F9BA15703C6

Create a list files for the MongoDBCreate a list files for the MongoDBCreate a list files for the MongoDBCreate a list files for the MongoDB

echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4

multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list

Install the MongoDInstall the MongoDInstall the MongoDInstall the MongoDB PackagesB PackagesB PackagesB Packages

sudo apt-get install -y mongodb

Page 2: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

We need to specify the services befor starting mongoDB

SSSStart MongoDBtart MongoDBtart MongoDBtart MongoDB sudo systemctl start mongodb

Page 3: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

Start MongoDB ShellStart MongoDB ShellStart MongoDB ShellStart MongoDB Shell

Mongo –shell

Try db to check the available databaseTry db to check the available databaseTry db to check the available databaseTry db to check the available database

Show db will show all the database in the MongoDB

To use particular database you need to write command To use particular database you need to write command To use particular database you need to write command To use particular database you need to write command

Use database_name

Page 4: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

CRUD

CREATE

Insert a collection user with record {name:”sagar”,age:26,status:”pending”}

READREADREADREAD

UPDATEUPDATEUPDATEUPDATE

We can see that user collection has two record with auto generated _id field and given status as

“pending” and “approved”

Page 5: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

Last command on MongoDB Shell shows our update statement.

We need to update user status as reject whose age is greater then 18

Similar to SQLSimilar to SQLSimilar to SQLSimilar to SQL : UPDATE user SET status=’reject’ WHERE age>18

Equivalent MongoDB Statement :Equivalent MongoDB Statement :Equivalent MongoDB Statement :Equivalent MongoDB Statement :

db.user.update({age:{$gt:18}},{$set:{status: ”reject”}})

You can see the output of update statement that modified records are 1, also the the READ commad

result that record that age>18age>18age>18age>18 (age=26age=26age=26age=26)))) is now having rejectrejectrejectreject statusstatusstatusstatus

Page 6: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

DELETEDELETEDELETEDELETE

To perform the remove command we inserted one more records with approvedapprovedapprovedapproved status, so now we have

total two record.

We can easily remove record based on WHERE conditionWe can easily remove record based on WHERE conditionWe can easily remove record based on WHERE conditionWe can easily remove record based on WHERE condition

db.user.remove({status:”approved”})

Page 7: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

HIVEHIVEHIVEHIVE

Download any stable apache HIVE tar.gz file, here I download 2.1.1 version

Untar the package to appropriate location. I untar and move the folder to /usr/local/hive/ directory so

that all my Hadoop related packages are manage in /usr/local/ directory however directory location

doesn’t matter.

Edit Edit Edit Edit .bashrc file.bashrc file.bashrc file.bashrc file

Set HIVE_HOME and PATH environment variable

Page 8: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

Copy hive-env.sh.template file to hive-env.sh and config

HADOOP_SIZE,HADOOP_SIZE,HADOOP_SIZE,HADOOP_SIZE, HADOOP_HOME,HADOOP_HOME,HADOOP_HOME,HADOOP_HOME, HIVE_CONF_DIRHIVE_CONF_DIRHIVE_CONF_DIRHIVE_CONF_DIR variable

Check that HIVE is configured Check that HIVE is configured Check that HIVE is configured Check that HIVE is configured perfectlyperfectlyperfectlyperfectly

hive --version

Start the Hadoop services

Page 9: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

Create directory to store HIVE Data

Page 10: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

Check that hive is started Using command hive

Initialize the database schema using following Initialize the database schema using following Initialize the database schema using following Initialize the database schema using following commandcommandcommandcommand

bin/schematool -initSchema -dbType derby

Page 11: Lab4 2HiveMongDB Sagar - Cleveland State Universitycis.csuohio.edu/~sschung/cis612/Lab4_2HiveMongDB_Sagar.pdf · Microsoft Word - Lab4_2HiveMongDB_Sagar Author: Sunnie Created Date:

ASSIGNMENT 4.2

NoSQL Data Processing Systems

CheckCheckCheckCheck the database inti logthe database inti logthe database inti logthe database inti log

Fri Apr 21 00:57:38 EDT 2017: Booting Derby version The Apache Software Foundation - Apache Derby - 10.10.2.0 - (1582446): instance a816c00e-015b-8edf-61c7-00000383bac0 on database directory /home/hduser/metastore_db with class loader sun.misc.Launcher$AppClassLoader@73f9ac Loaded from file:/usr/local/hive/apache-hive-2.1.1-bin/lib/derby-10.10.2.0.jar java.vendor=Oracle Corporation java.runtime.version=1.8.0_121-b13 user.dir=/home/hduser os.name=Linux os.arch=i386 os.version=4.4.0-72-generic derby.system.home=null Database Class Loader started - derby.database.classpath=''

Ref.Ref.Ref.Ref.

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu

https://www.digitalocean.com/community/tutorials/how-to-install-mongodb-on-ubuntu-16-04

https://www.edureka.co/blog/apache-hive-installation-on-ubuntu