Mysql-porxy 读写分离与Tcpdumo抓包工具

一、mysql-proxy概念:

MySQL Proxy 可以做Mysql数据库代理,基于mysql主从复制之上,能够实现读写分离、负载均衡等功能,即可以实现数据写入在主服务器,而数据查询则去往从服务器进行查询,从而大大降低主服务器的负载,对提升整个mysq集群能力有很大的作用。

下载地址:http://dev.mysql.com/downloads/mysql-proxy/

二、配置mysql主从集群,详情见http://zhangshijie.blog.51cto.com/806066/1606071

请确保主从同步运行正常、且时间一致,然后在继续下一步。

实验环境:

主服务器:192.168.10.204  MariaDB 10.0.10

从服务器:192.168.10.205  MariaDB 10.0.10

mysql-proxy服务器:192.168.10.206

mysql-proxy版本:0.8.3

1、mysql-proxy安装:

[[email protected] ~]# tar xvf /root/mysql-proxy-0.8.3-linux-glibc2.3-x86-64bit.tar.gz  -C /usr/local/

[[email protected] local]# ln -sv mysql-proxy-0.8.3-linux-glibc2.3-x86-64bit mysql-proxy

2、添加mysql-proxy用户:

[[email protected] local]# useradd mysql-proxy

3、编辑开机脚本:

[[email protected] local]# vim /etc/rc.d/init.d/mysql-proxy

#!/bin/bash

#

# mysql-proxy This script starts and stops the mysql-proxy daemon

#

# chkconfig: - 78 30

# processname: mysql-proxy

# description: mysql-proxy is a proxy daemon for mysql

# Source function library.

. /etc/rc.d/init.d/functions

prog="/usr/local/mysql-proxy/bin/mysql-proxy"

# Source networking configuration.

if [ -f /etc/sysconfig/network ]; then

. /etc/sysconfig/network

fi

# Check that networking is up.

[ ${NETWORKING} = "no" ] && exit 0

# Set default mysql-proxy configuration.

ADMIN_USER="admin"

ADMIN_PASSWD="admin"

ADMIN_LUA_SCRIPT="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"

PROXY_OPTIONS="--daemon"

PROXY_PID=/var/run/mysql-proxy.pid

PROXY_USER="mysql-proxy"

# Source mysql-proxy configuration.

if [ -f /etc/sysconfig/mysql-proxy ]; then

. /etc/sysconfig/mysql-proxy

fi

RETVAL=0

start() {

echo -n $"Starting $prog: "

daemon $prog $PROXY_OPTIONS --pid-file=$PROXY_PID --proxy-address="$PROXY_ADDRESS" --user=$PROXY_USER --admin-username="$ADMIN_USER" --admin-lua-script="$ADMIN_LUA_SCRIPT" --admin-password="$ADMIN_PASSWORD"

RETVAL=$?

echo

if [ $RETVAL -eq 0 ]; then

touch /var/lock/subsys/mysql-proxy

fi

}

stop() {

echo -n $"Stopping $prog: "

killproc -p $PROXY_PID -d 3 $prog

RETVAL=$?

echo

if [ $RETVAL -eq 0 ]; then

rm -f /var/lock/subsys/mysql-proxy

rm -f $PROXY_PID

fi

}

# See how we were called.

case "$1" in

start)

start

;;

stop)

stop

;;

restart)

stop

start

;;

condrestart|try-restart)

if status -p $PROXY_PIDFILE $prog >&/dev/null; then

stop

start

fi

;;

status)

status -p $PROXY_PID $prog

;;

*)

echo "Usage: $0 {start|stop|restart|reload|status|condrestart|try-restart}"

RETVAL=1

;;

esac

exit $RETVAL

4、为服务脚本提供配置文件/etc/sysconfig/mysql-proxy,内容如下所示:

# Options for mysql-proxy

ADMIN_USER="admin"

ADMIN_PASSWORD="admin"

ADMIN_ADDRESS=""

ADMIN_LUA_SCRIPT="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"

PROXY_ADDRESS=""

PROXY_USER="mysql-proxy"

PROXY_OPTIONS="--daemon --log-level=info --log-use-syslog --plugins=proxy --plugins=admin --proxy-backend-addresses=192.168.10.204:3306 --proxy-read-only-backend-addresses=192.168.10.205:3306 --proxy-lua-script=/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua"

5、 mysql-proxy的配置选项

mysql-proxy的配置选项大致可分为帮助选项、管理选项、代理选项及应用程序选项几类:

--help

--help-admin

--help-proxy

--help-all ———— 以上四个选项均用于获取帮助信息;

--proxy-address=host:port ———— 代理服务监听的地址和端口;

--admin-address=host:port ———— 管理模块监听的地址和端口;

--proxy-backend-addresses=host:port ———— 后端mysql服务器的地址和端口;

--proxy-read-only-backend-addresses=host:port ———— 后端只读mysql服务器的地址和端口;

--proxy-lua-script=file_name ———— 完成mysql代理功能的Lua脚本;

--daemon ———— 以守护进程模式启动mysql-proxy;

--keepalive ———— 在mysql-proxy崩溃时尝试重启之;

--log-file=/path/to/log_file_name ———— 日志文件名称;

--log-level=level ———— 日志级别;

--log-use-syslog ———— 基于syslog记录日志;

--plugins=plugin,.. ———— 在mysql-proxy启动时加载的插件;

--user=user_name ———— 运行mysql-proxy进程的用户;

--defaults-file=/path/to/conf_file_name ———— 默认使用的配置文件路径;其配置段使用[mysql-proxy]标识;

--proxy-skip-profiling ———— 禁用profile;

--pid-file=/path/to/pid_file_name ———— 进程文件名;

6、创建admin.lua文件,将其保存至/usr/local/mysql-proxy/share/doc/mysql-proxy/目录中

[[email protected] local]# vim /usr/local/mysql-proxy/share/doc/mysql-proxy/load-multi.lua

--[[ $%BEGINLICENSE%$

Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.

This program is free software; you can redistribute it and/or

modify it under the terms of the GNU General Public License as

published by the Free Software Foundation; version 2 of the

License.

This program is distributed in the hope that it will be useful,

but WITHOUT ANY WARRANTY; without even the implied warranty of

MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

GNU General Public License for more details.

You should have received a copy of the GNU General Public License

along with this program; if not, write to the Free Software

Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA

02110-1301  USA

$%ENDLICENSE%$ --]]

---

--[[

(Not so) simple example of a run-time module loader for MySQL Proxy

Usage:

1. load this module in the Proxy.

2. From a client, run the command

PLOAD name_of_script

and you can use the features implemented in the new script

immediately.

3. Successive calls to PLOAD will load a new one.

4. Previous scripts will still be active, and the loader will

use all of them in sequence to see if one handles the

request

5. CAVEAT. If your script has a read_query function that

*always* returns a non null value (e.g. a logging feature)

then other modules, although loaded, are not used.

If you have any such modules, you must load them at the

end of the sequence.

6. to remove a module, use

UNLOAD module_name

IMPORTANT NOTICE:

the proxy will try to load the file in the directory where

the proxy (NOT THE CLIENT) was started.

To use modules not in such directory, you need to

specify an absolute path.

--]]

local VERSION = ‘0.1.3‘

local DEBUG = os.getenv(‘DEBUG‘) or 0

DEBUG = DEBUG + 0

---

-- print_debug()

-- conditionally print a message

--

function proxy.global.print_debug (msg)

if DEBUG > 0 then

print(msg)

end

end

---

-- Handlers for MySQL Proxy hooks

-- Initially, they are void.

-- If the loaded module has implemented any

-- functions, they are associated with these handlers

--

if proxy.global.loaded_handlers == nil then

proxy.global.loaded_handlers = {

rq   = {},

rqr  = {},

dc   = {},

cs   = {},

rh   = {},

ra   = {},

rar  = {},

}

end

---

-- list of functions loaded from user modules

--

if proxy.global.handled_function == nil then

proxy.global.handled_functions = {

rq  = ‘read_query‘,

rqr = ‘read_query_result‘,

cs  = ‘connect_server‘,

rh  = ‘read_handshake‘,

ra  = ‘read_auth‘,

rar = ‘read_auth_result‘,

dc  = ‘disconnect_client‘,

}

end

if proxy.global.handler_status == nil then

proxy.global.handler_status = {}

end

local funcs_to_create = {

cs  = 0,

ra  = 1,

rar = 1,

rh  = 1,

dc  = 0,

}

---

-- creates the hooks for Proxy handled functions

--

for id, has_param in pairs(funcs_to_create) do

local parameter = has_param == 1 and ‘myarg‘ or ‘‘

local fstr = string.format(

[[

function %s(%s)

if #proxy.global.loaded_handlers[‘%s‘] then

for i, h in pairs(proxy.global.loaded_handlers[‘%s‘])

do

if h then

proxy.global.print_debug ( ‘handling "%s" using ‘ .. h.name)

local result = h.handler(%s)

if result then

return result

end

end

end

end

-- return

end

]],

proxy.global.handled_functions[id],

parameter,

id, id, proxy.global.handled_functions[id], parameter

)

proxy.global.print_debug (‘creating function ‘ .. proxy.global.handled_functions[id])

assert(loadstring(fstr))()

end

---

-- an error string to describe the error occurring

--

local load_multi_error_str = ‘‘

---

-- set_error()

--

-- sets the error string

--

-- @param msg the message to be assigned to the error string

--

function set_error(msg)

load_multi_error_str = msg

proxy.global.print_debug (msg)

return nil

end

---

-- file_exists()

--

-- checks if a file exists

--

-- @param fname the file name

--

function file_exists(fname)

local fh=io.open( fname, ‘r‘)

if fh then

fh:close()

return true

else

return false

end

end

local module_counter = 0

---

--  get_module()

--

--  This function scans an existing lua file, and turns it into

--  a closure exporting two function handlers, one for

--  read_query() and one for read_query_result().

--  If the input script does not contain the above functions,

--  get_module fails.

--

--      @param module_name the name of the existing script

--

function get_module(module_name)

--

-- assumes success

--

load_multi_error_str = ‘‘

--

-- the module is copied to a temporary file

-- on a given directory

--

module_counter = module_counter + 1

local new_module = ‘tmpmodule‘ .. module_counter

local tmp_dir   = ‘/tmp/‘

local new_filename = tmp_dir .. new_module .. ‘.lua‘

local source_script = module_name

if not source_script:match(‘.lua$‘) then

source_script = source_script .. ‘.lua‘

end

--

-- if the new module does not exist

-- an error is returned

--

if not file_exists(source_script) then

set_error(‘file not found ‘ .. source_script)

return

end

--

-- if the module directory is not on the search path,

-- we need to add it

--

if not package.path:match(tmp_dir) then

package.path = tmp_dir .. ‘?.lua;‘ .. package.path

end

--

-- Make sure that the module is not loaded.

-- If we don‘t remove it from the list of loaded modules,

-- subsequent load attempts will silently fail

--

package.loaded[new_module] = nil

local ofh = io.open(new_filename, ‘w‘)

--

-- Writing to the new package, starting with the

-- header and the wrapper function

--

ofh:write( string.format(

"module(‘%s‘, package.seeall)\n"

.. "function make_funcs()\n" , new_module)

)

local found_funcs = {}

--

-- Copying contents from the original script

-- to the new module, and checking for the existence

-- of the handler functions

--

for line in io.lines(source_script) do

ofh:write(line .. "\n")

for i,v in pairs(proxy.global.handled_functions) do

if line:match(‘^%s*function%s+‘ ..v .. ‘%s*%(‘) then

found_funcs[i] = v

break

end

end

end

--

-- closing the wrapper on the new module

--

local return_value = ‘‘

for i,v in pairs(found_funcs) do

return_value = return_value .. i .. ‘ = ‘ .. v .. ‘, ‘

end

ofh:write(

‘return { ‘ ..  return_value ..  ‘}\n‘ ..

‘end\n‘

)

ofh:close()

--

-- Final check. If the handlers were not found, the load fails

--

--

if (found_one == false ) then

set_error(‘script ‘ .. source_script ..

‘ does not contain a proxy handled function‘)

return

end

--

-- All set. The new module is loaded

-- with a new function that will return the handlers

--

local result = require(new_module)

os.remove(new_filename)

return result

end

---

-- simple_dataset()

--

-- returns a dataset made of a header and a message

--

-- @param header the column name

-- @param message the contents of the column

function proxy.global.simple_dataset (header, message)

proxy.response.type = proxy.MYSQLD_PACKET_OK

proxy.response.resultset = {

fields = {{type = proxy.MYSQL_TYPE_STRING, name = header}},

rows = { { message} }

}

return proxy.PROXY_SEND_RESULT

end

---

-- make_regexp_from_command()

--

-- creates a regular expression for fast scanning of the command

--

-- @param cmd the command to be converted to regexp

--

function proxy.global.make_regexp_from_command(cmd, options)

local regexp= ‘^%s*‘;

for ch in cmd:gmatch(‘(.)‘) do

regexp = regexp .. ‘[‘ .. ch:upper() .. ch:lower() .. ‘]‘

end

if options and options.capture then

regexp = regexp  .. ‘%s+(%S+)‘

end

return regexp

end

--

-- The default command for loading a new module is PLOAD

-- You may change it through an environment variable

--

local proxy_load_command    = os.getenv(‘PLOAD‘)  or ‘pload‘

local proxy_unload_command  = os.getenv(‘PUNLOAD‘)  or ‘punload‘

local proxy_help_command    = os.getenv(‘PLOAD_HELP‘)  or ‘pload_help‘

local currently_using       = 0

local pload_regexp          = proxy.global.make_regexp_from_command(proxy_load_command, {capture = 1})

local punload_regexp        = proxy.global.make_regexp_from_command(proxy_unload_command,{ capture = 1} )

local pload_help_regexp     = proxy.global.make_regexp_from_command(proxy_help_command,{ capture = nil} )

local pload_help_dataset    = {

{proxy_load_command   .. ‘ module_name‘,   ‘loads a given module‘},

{proxy_unload_command .. ‘module_name‘,    ‘unloads and existing module‘},

{proxy_help_command,    ‘shows this help‘},

}

---

-- removes a module from the loaded list

--

function remove_module (module_name)

local found_module = false

local to_delete = { loaded = {}, status = {} }

for i,lmodule in pairs(proxy.global.handler_status) do

if i == module_name then

found_module = true

local counter = 0

for j,h in pairs(lmodule) do

-- proxy.global.print_debug(‘removing ‘.. module_name .. ‘ (‘ .. i ..‘) ‘ .. h.id .. ‘ -> ‘ .. h.ndx )

to_delete[‘loaded‘][h.id] = h.ndx

counter = counter + 1

to_delete[‘status‘][i] = counter

end

end

end

for i,v in pairs (to_delete[‘loaded‘]) do

table.remove(proxy.global.loaded_handlers[i], v)

end

for i,v in pairs (to_delete[‘status‘]) do

table.remove(proxy.global.handler_status[i], v)

end

if found_module == false then

return proxy.global.simple_dataset(module_name, ‘NOT FOUND‘)

end

return proxy.global.simple_dataset(module_name, ‘unloaded‘)

end

---

-- creates a dataset from a list of header names

-- and a list of rows

-- @param header a list of field names

-- @param dataset a list of row contents

function proxy.global.make_dataset (header, dataset)

proxy.response.type = proxy.MYSQLD_PACKET_OK

proxy.response.resultset = {

fields = {},

rows = {}

}

for i,v in pairs (header) do

table.insert(proxy.response.resultset.fields, {type = proxy.MYSQL_TYPE_STRING, name = v})

end

for i,v in pairs (dataset) do

table.insert(proxy.response.resultset.rows, v )

end

return proxy.PROXY_SEND_RESULT

end

--

-- This function is called at each query.

-- The request for loading a new script is handled here

--

function read_query (packet)

currently_using = 0

if packet:byte() ~= proxy.COM_QUERY then

return

end

local query = packet:sub(2)

-- Checks if a PLOAD command was issued.

-- A regular expresion check is faster than

-- doing a full tokenization. (Especially on large queries)

--

if (query:match(pload_help_regexp)) then

return proxy.global.make_dataset({‘command‘,‘description‘}, pload_help_dataset)

end

local unload_module = query:match(punload_regexp)

if (unload_module) then

return remove_module(unload_module)

end

local new_module = query:match(pload_regexp)

if (new_module) then

--[[

If a request for loading is received, then

we attempt to load the new module using the

get_module() function

--]]

local new_tablespace = get_module(new_module)

if (new_tablespace) then

local handlers = new_tablespace.make_funcs()

--

-- The new module must have at least  handlers for read_query()

-- or disconnect_client. read_query_result() is optional.

-- The loading function returns nil if no handlers were found

--

proxy.global.print_debug(‘‘)

proxy.global.handler_status[new_module] = {}

for i,v in pairs( proxy.global.handled_functions) do

local handler_str = type(handlers[i])

proxy.global.print_debug (i .. ‘ ‘ .. handler_str )

if handlers[i] then

table.insert(proxy.global.loaded_handlers[i] ,

{ name = new_module, handler = handlers[i]} )

table.insert(proxy.global.handler_status[new_module],

{ func = proxy.global.handled_functions[i], id=i, ndx = #proxy.global.loaded_handlers[i] })

end

end

if handlers[‘rqr‘] and not handlers[‘rq‘] then

table.insert(proxy.global.loaded_handlers[‘rq‘] , nil )

end

if handlers[‘rq‘] and not handlers[‘rqr‘] then

table.insert(proxy.global.loaded_handlers[‘rqr‘] , nil )

end

--

-- Returns a confirmation that a new  module was loaded

--

return proxy.global.simple_dataset(‘info‘, ‘module "‘ .. new_module .. ‘" loaded‘ )

else

--

-- The load was not successful.

-- Inform the user

--

return proxy.global.simple_dataset(‘ERROR‘, ‘load of "‘

.. new_module .. ‘" failed (‘

.. load_multi_error_str .. ‘)‘ )

end

end

--

-- If a handler was installed from a new module, it is called

-- now.

--

if #proxy.global.loaded_handlers[‘rq‘] then

for i,rqh in pairs(proxy.global.loaded_handlers[‘rq‘])

do

proxy.global.print_debug ( ‘handling "read_query" using ‘ .. i .. ‘ -> ‘ .. rqh.name)

local result = rqh.handler(packet)

if (result) then

currently_using = i

return result

end

end

end

end

--

-- this function is called every time a result set is

-- returned after an injected query

--

function read_query_result(inj)

--

-- If the dynamically loaded module had an handler for read_query_result()

-- it is called now

--

local rqrh = proxy.global.loaded_handlers[‘rqr‘][currently_using]

if rqrh then

proxy.global.print_debug ( ‘handling "read_query_result" using ‘ .. currently_using .. ‘ -> ‘ .. rqrh.name)

local result = rqrh.handler(inj)

if result then

return result

end

end

end

7、添加到启动列表设置开机启动并启动服务:

[[email protected] local]# chkconfig  --add mysql-proxy

[[email protected] local]# chkconfig  mysql-proxy on

[[email protected] local]# service mysql-proxy start

三:验证:

1、端口验证:

2、管理功能验证:

主从服务器创建远程账号:

mysql> grant all  on *.* to ‘admin‘@‘192.168.%.%‘ identified by ‘123456‘;

mysql> grant all  on *.* to ‘admin‘@‘192.168.%.%‘ identified by ‘123456‘;

使用服务器账号在mysql-proxy服务器登陆:

[[email protected] local]# mysql -uadmin -padmin -h192.168.10.206 --port 4041

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 1

Server version: 5.0.99-agent-admin

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql> SELECT * FROM backends;

+-------------+---------------------+---------+------+------+-------------------+

| backend_ndx | address             | state   | type | uuid | connected_clients |

+-------------+---------------------+---------+------+------+-------------------+

|           1 | 192.168.10.204:3306 | unknown | rw   | NULL |                 0 |

|           2 | 192.168.10.205:3306 | unknown | ro   | NULL |                 0 |

+-------------+---------------------+---------+------+------+-------------------+

3、可以使用mysql-proxy连接从服务器查看mysql状态,也可以连接主服务器创建数据库和表以及插入数据:

[[email protected] local]# mysql -uadmin -padmin -h192.168.10.205  --port 3306

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 9

Server version: 5.5.5-10.0.10-MariaDB-log MariaDB Server

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

4. 读写分离测试

[[email protected] local]# mysql -uadmin -padmin -h192.168.10.206  --port 4041

四:使用tcpdumo抓包工具抓包:

要想抓包必须要启用网卡的混杂模式

-i:任意接口

-n:不反解IP地址为主机名

-nn:不反解IP地址为主机名及端口号到服务器名

-X:将数据包转换为ASCII格式方便查看

-XX:将数据包转换为ASCII格式方便查看并显示数据帧信息

-v,-vv,-vvv  详细信息

-c 指定获取的数据包数量

-s:定义字节捕获单元长度(大小),默认即使此选项,不需要单独指定

-S:显示TCP/IP序列号

-e:获以太网首部信息

-A:将捕获的报文以ASCII显示

tcpdump的语法:

tcpdump [options] [Protocol] [Direction] [Host(s)] [Value] [Logical Operations] [Other expression]

Protocol(协议):

Values(取值): ether, fddi, ip, arp, rarp, decnet, lat, sca, moprc, mopdl, tcp and udp.

If no protocol is specified, all the protocols are used.

Direction(流向):

Values(取值): src, dst, src and dst, src or dst

If no source or destination is specified, the "src or dst" keywords are applied. (默认是src or dst)

For example, "host 10.2.2.2" is equivalent to "src or dst host 10.2.2.2".

Host(s)(主机):

Values(替代关键字): net, port, host, portrange.

If no host(s) is specified, the "host" keyword is used. 默认如果此段没有指定关键字,默认即host。

For example, "src 10.1.1.1" is equivalent to "src host 10.1.1.1".

Logical Operations:

(1) AND

and or &&

(2) OR

or or ||

(3) EXCEPT

not or !

测试ftp抓包:

[[email protected] ~]# tcpdump  -i eth0 -nn -XX dst host 192.168.10.204 and tcp dst port 21

可以抓取到ftp明文的用户名密码,因此在公网一定不使用明文账户口令,尤其是mysql数据要使用ssl加密传输。

测试完成!!!!!!!!!!!!1

时间: 01-28