Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL
-
Upload
fujio-turner -
Category
Software
-
view
389 -
download
0
description
Transcript of Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL
![Page 1: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/1.jpg)
SQL TO ECL (ENTERPRISE CONTROL LANGUAGE)
By Fujio Turner
@FujioTurner
![Page 2: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/2.jpg)
WHAT IS ECLECL (Enterprise Control Language) is a C++ based query language for use with HPCC Systems Big Data platform. ECLs syntax and format is very simple and easy to learn.!!
Note - ECL is very similar to Hadoop’s pig ,but!more expressive and feature rich.
![Page 3: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/3.jpg)
JDBC CONNECTOR
SELECT * FROM `allPeople`
HPCC Systems Can Do SQL
Learn ECL to Do Complex Queries
SQL
OR
Graph
http://www.slideshare.net/hpccsystems/jdbc-hpcc
![Page 4: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/4.jpg)
1.
2.
3.
4.
Lets Practice ECL
![Page 5: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/5.jpg)
SCHEMA
Layout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; END;
CREATE TABLE layout_person ( PersonID INT(15) NOT NULL, FirstName VARCHAR(15) NOT NULL, LastName VARCHAR(25) NOT NULL, PRIMARY KEY (PersonID) );
SQL ECL
Apply Schema on Read
![Page 6: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/6.jpg)
DATAallPeople := DATASET([ {1,'Fred','Smith'}, {2,'Joe','Blow'}, {3,’Jane','Smith'} ],Layout_Person);
USE ‘layout_person’;SQL ECL
allPeople := DATASET(‘~file’,Layout_Person,THOR);
or
Inline Data
Data from HPCC Systems Cluster
![Page 7: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/7.jpg)
WHERELayout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; END; !allPeople := DATASET([ {1,'Fred','Smith'}, {2,'Joe','Blow'}, {3,’Jane','Smith'} ],Layout_Person); !somePeople := allPeople(LastName = 'Smith'); !OUTPUT(somePeople);
SELECT * FROM `allPeople` WHERE `LastName` = ‘Smith’
SQL ECL
![Page 8: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/8.jpg)
SELECTLayout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; END; !allPeople := DATASET([ {1,'Fred','Smith'}, {2,'Joe','Blow'}, {3,’Jane','Smith'} ],Layout_Person); !//somePeople := allPeople(LastName = 'Smith'); //OUTPUT(somePeople); !OUTPUT(allPeople);
SELECT * FROM `allPeople`
SQL ECL
![Page 9: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/9.jpg)
SELECT WITHLayout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; END; !allPeople := DATASET([ {1,'Fred','Smith'}, {2,'Joe','Blow'}, {3,’Jane','Smith'} ],Layout_Person); !OUTPUT(allPeople,{LastName,FirstName});
SELECT ‘LastName’,’FirstName’ FROM `allPeople`
SQL ECL
![Page 10: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/10.jpg)
ORDER BYLayout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; END; !allPeople := DATASET([ {1,'Fred','Smith'}, {2,'Joe','Blow'}, {3,’Jane','Smith'} ],Layout_Person); !!OUTPUT(SORT(allPeople,LastName));
SELECT * FROM `allPeople` ORDER BY ‘LastName’
SQL ECL
![Page 11: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/11.jpg)
GROUP BYLayout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; END; !allPeople := DATASET([ {1,'Fred','Smith'}, {2,'Joe','Blow'}, {3,’Jane','Smith'} ],Layout_Person); !!OUTPUT(GROUP(SORT(allPeople,LastName)));
SELECT * FROM `allPeople` GROUP BY ‘LastName’
SQL ECL
![Page 12: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/12.jpg)
GROUP BY & COUNTLayout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; END; !allPeople := DATASET([ {1,'Fred','Smith'}, {2,'Joe','Blow'}, {3,’Jane','Smith'} ],Layout_Person); !!OUTPUT(TABLE(allPeople, {LastName, LastNameC := COUNT(GROUP)}, LastName) );
SELECT LastName,COUNT(*) FROM `allPeople` GROUP BY ‘LastName’
SQL ECL
![Page 13: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/13.jpg)
LIKEIMPORT Std; !Layout_Person := RECORD UNSIGNED1 PersonID; STRING15 FirstName; STRING25 LastName; STRING33 Email; END; !allPeople := DATASET([ {1,’Fred’,’Smith’,’[email protected]’}, {2,'Joe','Blow',’[email protected]’}, {3,’Jane’,’Smith',’[email protected]’} ],Layout_Person); !emails := allPeople( Std.Str.EndsWith(Email, ‘@gmail.com’) ); OUTPUT(emails);
SELECT * FROM `allPeople` WHERE `Email` LIKE ‘%@gmail.com’
SQL ECL
Import C++ libraries and/or make
your own functions & libraries
![Page 14: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/14.jpg)
MORE SAMPLESInstall HPCC Systems and go to the admin playground
and see more samples of ECL.
![Page 15: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/15.jpg)
ECL LANGUAGE GUIDEhttp://hpccsystems.com/download/docs/ecl-language-referenceJOIN!
MERGE!LENGTH!REGEX!
ROUND!SUM!
COUNT!TRIM!WHEN!
AVE!ABS!
CASE!DEDUP!
NORMALIZE!DENORMALIZE!
IF!SORT!
GROUP!more ….
![Page 16: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/16.jpg)
SCHEMA MADE EZ
http://hpccsystems.com/demos/data-profiling-demo
CSV IN
Schema OUT
Click
Uploading data from a new source and want a quick schema?
Click below
![Page 18: Meetup - Exabyte Big Data - HPCC Systems - SQL to ECL](https://reader033.fdocuments.us/reader033/viewer/2022051818/54b718a54a7959af718b465f/html5/thumbnails/18.jpg)
http://www.youtube.com/watch?v=8SV43DCUqJg
Watch how to install HPCC Systems
in 5 Minutes
Download HPCC Systems Open Source
Community Edition
or
Source Codehttps://github.com/hpcc-systems
http://hpccsystems.com/download/