Introducing Lua for MySQL Proxy - datacharmer.comdatacharmer.com/downloads/introducing_lua.pdf ·...
Transcript of Introducing Lua for MySQL Proxy - datacharmer.comdatacharmer.com/downloads/introducing_lua.pdf ·...
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Giuseppe MaxiaQA Developer
December 2007
Introducing Lua for MySQL Proxy
programming
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Giuseppe Maxia
MySQL QA Developer
a.k.a. The Data Charmer
✦ Despite popular belief, not a Lua expert (less than six months experience)
About me
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
• MySQL Proxy architecture (very quickly)• Introducing Lua• Language fundamentals• Examples• Comments
Agenda
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
• MySQL Forge • http://forge.mysql.com/wiki/MySQL_Proxy
• Getting started article• http://dev.mysql.com/tech-resources/articles/proxy-gettingstarted.html
• Lua• http://www.lua.org/docs.html• http://lua-users.org/wiki/TutorialDirectory
References
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Proxy (< lat. procuratio)
5
= Someone taking care of someone else's interests
A server proxy is something acting on behalf of another server
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Proxy Overview
6
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Proxy Overview
7
PROXY CORE
connection hook
read query hook
read result hook
function
Lua script
function
functionfunctionfunction
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Lua Overview
8
??Why not ...{Perl ?
PHP?Javascript?[whatever]?
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Lua Overview
9
• SMALL ( < 200 KB)• DESIGNED for
EMBEDDED systems• Widely used (lighttpd)
lighttpd, like MySQL Proxy, was created by Jan Kneschke
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Lua Overview
10
Very popular among game writers
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Proxy Overview
11
connect_server
Lua script
read_auth
read_auth_result
read_handshake
read_query
read_query_result
disconnect_client
global context
session contextsession
contextsession contextsession
contextsession contextsession
contextsession context
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Using Lua files
12
/usr/local/sbin/mysql-proxy \ --proxy-lua-script=/path/name.lua
IMPORTANT!THE SCRIPT DOES NOT START UNTIL THE
FIRST CLIENT CONNECTION
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Lua types
13
• nil• number• string• table• function• userdata
a = nilb = 1c = 'abc't = { a,b,c }f = printu = some_C_struct
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Lua comments
14
-- simple comment
print(1)--[[print(2)print('hello')--]]print(3)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Lua comments
15
-- simple comment
--[=[ print(1)--[[print(2)print('hello')--]]print(3)--]=]
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
numbers and strings
16
• nil = no value (false)• number = floating point• '5' =/= 5 a = nil
b = 5; c = '5'print (b == c)falseprint (b == c +0)true
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
numbers and strings
17
• conversion on demand
a = 5 ; b = '5'
print(type(a), type(b))number stringprint(type(b+0))numberprint(type(a .. ""))string
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
numbers and strings
18
• conversion on demand
a = 5 ; b = '5'
print(type(tostring(a)))stringprint(type(tonumber(b)))number
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
strings
19
• smart quoting
a = 'Hello' b = "World"c = "Can't"d = [[Don't say "Hello"]]e = [=["d'oh" [[braces]]!]=]
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
tables
20
• associative arrays• can be used as arrays• can create complex
structurest1 = {10, 20, 30 }t2 = { a = 'abc', b = 2, c = { 3, 4}}
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
functions
21
• can be assigned to variables
• new functions can override existing ones
function f (x) print(x)endg = fg(10)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
userdata
22
• containers to exchange data between Lua and host language
• can have "tag methods"
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Statements
23
• normal assignmentsa = 3b = get_num() -- func return
• multiple assignmentsa,b = 3,2
• multiple return valuesfunction x () return 1, 'OK'end
a, b, c = x() -- a = 1, b = 'OK', c = nil
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
statement blocks
24
• ifif condition then statementsend
• whilewhile condition do statementsend
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
statement blocks
25
• forfor var = 1, 10 [,step] do statementsend
• forfor n,v in pairs(table_var) do statementsend
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
sample function
26
• read_query
1 function read_query(packet) 2 if packet:byte() ~= 3 proxy.PROXY_COM_QUERY 4 then 5 return 6 end 7 local query = packet:sub(2) 8 print('received ' .. query) 9 end
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
some details
27
== equals ~= not equal string.byte(packet) packet:byte()
string.sub(packet,2) packet:sub(2) 'abc' .. '123' == 'abc123'
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Using tables
28
t = {}t[1] = 'a' --First element 1, ! 0t[2] = 'b'table.insert(t, 'c') -- or t[ #t +1 ] = 'c'
t = {'a', 'b', 'c' }t = {1 = 'a', 2 = 'b', 3 = 'c'}print (t[2])b
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Using tables
29
sizes = {}sizes['john'] = 'XL'sizes['paul'] = 'M'sizes['fred'] = 'L'
sizes = { john = 'XL', paul = 'M', fred = 'L',}
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Using tables
30
sizes = { john = 'XL', paul = 'M', fred = 'L',} print(sizes['john'])XLprint(sizes.paul)M
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Using tables
31
for i, v in ipairs(t) do print (i ..' -> ' .. v)end
for name,size in pairs(sizes) do print(name .. ' ' .. wears .. ' ' .. size)end
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Watch out
32
/* C / C++ */ int a = 0;printf("%s\n", a ? "true" : "false");false
-- Luaa = 0print ( a and "true" or "false")true
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Watch out
33
-- Luaa = falseprint ( a and "true" or "false")false
a = nilprint ( a and "true" or "false")false
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Finding text
34
query = 'SELECT id FROM t1'
local cmd, column = query:match("(SELECT)%s+(%w+)")
if cmd then -- do something with queryend
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Finding text
35
• Regular expressions• similar to Perl/PHP, but simpler
• % instead of \• (captures)• [character classes]• ^ $ + - ? * • no alternation (a|b)• no modifiers /i
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Finding text (Proxy way)
36
local tk = require('proxy.tokenizer')
local tokens = tk.tokenize(query)
if tokens[1].token_name == 'TK_SQL_SELECT' then -- do something with queryend
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Finding text (Proxy way)
37
-- each token is a table
token = { token_name = 'TK_SQL_SELECT', text = 'select', token_id = 204}
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
I/O
38
-- files are objects
local fname = '/tmp/test.txt'assert(fh = io.open(fname,'r'))
for line in fh:lines() do print(line)endfh:close()
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
I/O
39
-- files are objects
local fname = '/tmp/test.txt'
assert(fh = io.open(fname,'w'))for x = 1, 100 do fh:write('new row ' .. x) fh:flush()endfh:close()
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Examples
40
http://forge.mysql.com/wiki/Lua_Scripts_For_MySQL_Proxy_Examples
• all hooks• session bandwidth• user bandwidth• blocking commands
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
all-hooks (1)
41
source: all-hooks.lua
function read_auth( auth )...function connect_server()...function read_handshake( auth )...function read_auth_result( auth )...function disconnect_client()...function read_query (packet)...function read_query_result (inj)...
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
all-hooks (2)
42
source: all-hooks.lua
local access_ndx = 0
function print_access(msg) access_ndx = access_ndx + 1 print( string.format('%3d %-30s',access_ndx,msg))end
function read_auth( auth ) print_access ('inside read_auth ')end
function connect_server() print_access ('inside connect_server')end
function read_handshake( auth ) print_access ('inside read_handshake' )end-- ... to be continued
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
all-hooks (3)
43
source: all-hooks.lua
function read_auth_result( auth ) print_access ('inside read_auth_result ')end
function read_query (packet) print_access('inside read_query \t' .. packet:sub(2)) proxy.queries:append(1, packet) return proxy.PROXY_SEND_QUERYend
function read_query_result (inj) print_access('inside read_query_result \t' .. inj.query)end
function disconnect_client() print_access('inside disconnect_client')end
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
read_query & read_query_result
44
client
MySQL Proxy
SERVER
query
functionread_query
function
read_query_result
query
result
result
if a query is passed directly to the server,its result is NOT evaluated by read_query_result
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
read_query & read_query_result
45
client
MySQL Proxy
SERVER
query
function read_query
functionread_query_result
query queue
query
result
result
only if a query is added to the query queue,its result is evaluated by read_query_result
queryquery
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
all-hooks (4)
46
source: all-hooks.lua
sample output/usr/local/sbin/mysql-proxy --proxy-lua-script=all-hooks.lua 1 inside connect_server 2 inside read_handshake 3 inside read_auth 4 inside read_auth_result 5 inside read_query select @@version_comment limit 1 6 inside read_query_result select @@version_comment limit 1 7 inside read_query select now() 8 inside read_query_result select now() 9 inside read_query 10 inside disconnect_client
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
session-bandwidth (1)
47
source: session-bandwidth.lua
local bandwidth = 0
function read_query (packet ) bandwidth = bandwidth + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERYend
-- to be continued ...
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
session-bandwidth (2)
48
source: session-bandwidth.lua
-- (continue)
function read_query_result(inj) local fields = inj.resultset.fields if (fields) then for i = 1, #fields do bandwidth = bandwidth + (fields[i] and fields[i].name:len() or 0) end for row in inj.resultset.rows do for i = 1, #inj.resultset.fields do bandwidth = bandwidth + (row[i] and row[i]:len() or 0) end end end print (proxy.connection.server['thread_id'],bandwidth)end
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
session-bandwidth
49
source: session-bandwidth.lua
sample output:
/usr/local/sbin/mysql-proxy --proxy-lua-script=bandwidth.lua64 7864 115
65 7865 115
66 7866 1118
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
user-bandwidth (1)
50
source: user-bandwidth.lua
proxy.global.bandwidth = proxy.global.bandwidth or {}
local session_user
function read_auth( auth ) session_user = auth.username proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] or 0end
-- to be continued ...
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
user-bandwidth (2)
51
source: user-bandwidth.lua
function read_query (packet ) proxy.global.bandwidth[session_user ] = proxy.global.bandwidth[session_user] + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERYend
-- to be continued ...
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 52
function read_query_result(inj) local fields = inj.resultset.fields local rows = inj.resultset.rows if fields then for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (fields[i] and fields[i].name:len() or 0) end if rows then for row in rows do for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (row[i] and row[i]:len() or 0) end end end end print (session_user .. ' -> ' .. proxy.global.bandwidth[session_user])end
user-bandwidth (3)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 53
source: user-bandwidth.lua
Sample session
gmax -> 376gmax -> 399
root -> 599root -> 10641
gmax -> 478gmax -> 20520
user-bandwidth (4)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 54
source: query-block.lua
local tokenizer = require("proxy.tokenizer")----- make_regexp_from_command()---- creates a regular expression for fast scanning of the command-- -- @param cmd the command to be converted to regexp--function make_regexp_from_command(cmd) local regexp= '^%s*'; for ch in cmd:gmatch('(.)') do regexp = regexp .. '[' .. ch:upper() .. ch:lower() .. ']' end return regexpend-- to be continued ...
query-block (1)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 55
source: query-block.lua
local SHOW_REGEXP = make_regexp_from_command('show')
queries_to_filter = { { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLE', 'STATUS'} , }, { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLES'} , },}
-- to be continued ...
query-block (2)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 56
source: query-block.lua
function error_result (msg) proxy.response = { type = proxy.MYSQLD_PACKET_ERR, errmsg = msg, errcode = 7777, sqlstate = 'X7777', } return proxy.PROXY_SEND_RESULTend
-- to be continued ...
query-block (3)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 57
function read_query( packet ) if (packet:byte() ~= proxy.COM_QUERY) then return end local query = packet:sub(2) for i,query_to_filter in pairs(queries_to_filter) do if query:match(query_to_filter.prefix) then local full_tokens = tokenizer.tokenize(query) local tokens = tokenizer.bare_tokens(full_tokens, true) local found = 0 local requested = #query_to_filter.keywords for j,keyword in pairs(query_to_filter.keywords) do for k, token in pairs(tokens) do if token:upper() == keyword then found = found + 1 break end end end-- to be continued ...
query-block (4)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 58
function read_query( packet ) -- ...
if found == requested then -- to be filtered off return error_result('command <' .. table.concat(query_to_filter.keywords,' ') .. '> is not allowed' ) end end endend
query-block (5)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 59
source: query-block.lua
sample output:
show tables;ERROR 7777 (X7777): command <SHOW TABLES> is not allowed
query-block (6)
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Read more
60
http://www.lua.org/docs.html
online Lua documentation
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Read more
61
http://www.inf.puc-rio.br/~roberto/pil2/
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Read more
62
http://www.wrox.com/WileyCDA/WroxTitle/productCd-0470069171.html
Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database
Giuseppe MaxiaQA Developer
December 2007
QUESTIONS
?