Dynamic Columns
-
Upload
oleksiy-kovyrin -
Category
Documents
-
view
218 -
download
0
Transcript of Dynamic Columns
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 1/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MySQL Conference & Expo 2011
Michael “Monty” Widenius [email protected] “Sanja” Byelkin [email protected]
MariaDBDynamic Columns
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 2/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
RDBMS doesn't solve all common problems
The (web) store problem:All items need: ID, Type, Price, Country, Manufacturer)
A T-Shirt has the following additional properties:
Size, color...A computer has the following additional properties:
CPU, MHz, memory, Watt...
There is no easy way to store many different types into arelational database!
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 3/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
RDBMS doesn't solve all common problems
One common solutions to this is:
● Store all the 'extra' columns in a BLOB in some format(HTML?)
● You need a lot of extra work to manipulate the blob● Hard to access column data (usually done in client)● Overhead in storage (especially when you use HTML)● All values are 'text'
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 4/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
RDBMS doesn't solve all common problems
Another common solution:● Create one table for all the 'extra' columns:CREATE TABLE extra
(id int auto_increment, extra_column_id char(10),
value varchar(255));
INSERT INTO items set type=“t-shirt”, price=10;
INSERT INTO extra (NULL, LAST_INSERT_ID(), “color”,
“Blue”),(NULL, LAST_INSERT_ID(), “Size”, “M”);
The problems with this approach is:●
Every access to an extra column requires a key/rowlookup
● Slow performance (if database is not optimized for this)● Big overhead in storage (especially in index)
● Risk for errors as data is not typed
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 5/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns
Dynamic columns is a bridge between relationaldatabases and non relational databases
● With dynamic columns all extra columns are stored in apacked blob, maintained by the database.
● You can add more columns, remove or query them for arow.
● You can access columns in the server or retrieve the fullblob to the client and manipulate it there.
●
You can use virtual columns to create indexes on somevalues.
●True indexes for dynamic columns is planned for later.● Implemented through functions for use by ODBC, & etc.
● First implementation uses integer to access columns.
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 6/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: supportedtypes
● unsigned int
● int
● char [character set <cs>]
●
double● decimal [(<len>, <frac>)]
● time
● date
● datetime
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 7/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Creating a table with a dynamic column for the store:
CREATE TABLE item (
ID int auto_increment primary key,
Type_id int,Price decimal(7,2),
Country_id int,
Manufacturer_id int,
extra blob);
Where column 'extra' is dedicated to store dynamiccolumns. It could be any column able carry text.
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 8/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Creating/initializing a dynamic_column:
COLUMN_CREATE(column_nr, value [as type],
[column_nr, value [as type]], ...)
INSERT into item values
(NULL, 1 /* T-shirt */, 10, 1 /* Germany */, 1 /* Nike */,
COLUMN_CREATE(1 /* color */, "Blue", 2 /* Size */, "M"));
INSERT into item values(NULL, 2 /* computer */, 1000, 1 /* Germany */, 2 /* intel
*/,
COLUMN_CREATE(3 /* cpu */, "T9400", 5 /* MHz */, 800 as
unsigned int));
* *
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 9/18
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 10/18
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Deleting a dynamic column (if it exists):
COLUMN_DELETE(string, column_nr1,
column_nr2, ...);
UPDATE item SET extra=
COLUMN_DELETE(extra, 6)
WHERE id=2;
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 11/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Querying a dynamic column:
COLUMN_EXISTS(string, column_nr);SELECT
ID, Type_id, Price, Country_id, Manufacturer_id from itemwhere COLUMN_EXISTS(extra, 3);
Querying which columns exist:
COLUMN_LIST(string);SELECT COLUMN_LIST(extra) FROM item WHERE id=1;
“→ 1,2”
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 12/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax
Retrieving a dynamic column:
COLUMN_GET(column_nr, string as type);
SELECT id, COLUMN_GET( 1 /* color*/, extra as char)from item;
→ 1 Blue
→ 2 NULL
You can of course also do things like:SELECT id, COLUMN_GET(1, extra as char) FROM item
where Type_id=1
order by COLUMN_GET(1, extra as char);
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 13/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: C library
C Library allows the same manipulations with dynamiccolumns on the client side.
A description can be found in the worklog
http://askmonty.org/worklog/Server-Sprint/?tid=34 and in thesource.
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 14/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: encoding
How is the dynamic column encoded?
Header:<flag><number_of_columns>
Sorted index:<column_nr1><offset/type><column_nr2><offset/type>
Each column is stored as:<data1><data2>
Where 'offset' is offset from beginning of data part, 'type' is3 bits in offset.
Length of the data could be calculated by offsets of 2
neighbor fields.
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 15/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: data encoding
● Unsigned integer is just a variable integer field.● Signed integer coded to make the variable size efficient:
0 → 0
-1 → 1
1 → 2
-2 → 3
2 → 4
...
● Double, date, time, and date time are fixed-size fields●
String stores collation number and the string● Decimal stores sizes of parts before and after decimal point
and decimal in MySQL format.● NULL – means removing the field.
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 16/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: avaliability
When will dynamic columns be available?
First version is already pushed in separate tree for testing.
lp:~maria-captains/maria/5.3-mwl34
Should be available in main MariaDB 5.3 within severalweeks.
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 17/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: plans
● Adding name directory● Adding functional indices●
Supporting popular NoSQL data exchange formats(for example, JSON or XML)● Adding engine support for NoSQL databases asHBase
8/7/2019 Dynamic Columns
http://slidepdf.com/reader/full/dynamic-columns 18/18
Thanks
Q & A